Oct 30 18:33:40.288465 ip-10-0-140-122 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 30 18:33:40.288474 ip-10-0-140-122 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 30 18:33:40.288482 ip-10-0-140-122 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 30 18:33:40.288698 ip-10-0-140-122 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 30 18:33:50.407521 ip-10-0-140-122 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 30 18:33:50.407537 ip-10-0-140-122 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot f634fc995ea34a46bf49221d2d807091 --
Oct 30 18:36:00.102857 ip-10-0-140-122 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 18:36:00.487098 ip-10-0-140-122 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:36:00.487098 ip-10-0-140-122 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 18:36:00.487098 ip-10-0-140-122 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:36:00.487098 ip-10-0-140-122 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:36:00.493813 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493737    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:00.493813 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493812    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:00.493813 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493816    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493819    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493822    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493825    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493828    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493832    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493835    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493838    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493840    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493843    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493846    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493848    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493851    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493853    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493856    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493859    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493861    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493864    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493867    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493874    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:00.493930 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493877    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493880    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493883    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493885    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493888    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493890    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493893    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493896    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493898    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493901    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493904    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493907    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493910    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493912    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493915    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493918    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493920    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493923    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493927    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:00.494393 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493932    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493935    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493938    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493941    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493943    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493946    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493949    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493951    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493954    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493956    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493959    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493968    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493972    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493975    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493979    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493983    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493986    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493989    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493992    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:00.494892 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493995    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.493998    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494001    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494003    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494006    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494009    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494012    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494014    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494017    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494020    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494023    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494026    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494028    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494031    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494034    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494036    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494039    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494041    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494044    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494046    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:00.495323 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494049    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494051    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494054    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494056    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494059    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494061    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494064    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494067    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494069    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494072    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494075    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494078    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494080    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494083    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494085    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494088    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494091    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494095    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494097    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:00.495815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494100    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494102    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494105    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494108    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494110    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494113    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494116    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494902    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494909    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494912    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494915    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494918    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494921    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494926    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494930    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494933    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494936    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494939    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494941    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:00.496259 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494944    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494947    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494950    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494952    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494955    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494958    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494960    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494963    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494965    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494968    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494971    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494974    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494976    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494979    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494983    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494985    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494988    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494991    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494993    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:00.496723 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494995    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.494998    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495001    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495003    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495006    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495009    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495011    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495014    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495016    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495019    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495021    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495024    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495027    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495030    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495033    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495036    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495041    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495044    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495047    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495050    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:00.497167 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495052    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495055    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495057    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495060    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495063    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495065    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495069    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495071    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495074    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495076    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495079    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495082    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495087    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495090    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495093    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495095    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495098    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495100    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495103    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495105    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:00.497661 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495108    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495110    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495113    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495115    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495118    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495121    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495123    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495126    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495128    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495131    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495134    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495136    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495139    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495142    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495144    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495147    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495150    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495152    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495155    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495158    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:00.498133 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495160    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495162    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495165    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495167    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495170    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495172    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495175    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495178    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495180    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495184    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495186    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495189    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495191    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495194    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495196    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495707    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495712    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495715    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495717    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:00.498596 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495720    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495723    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495732    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495735    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495737    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495740    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495743    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495745    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495749    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495751    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495754    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495757    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495760    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495762    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495765    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495767    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495770    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495772    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495775    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:00.499134 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495777    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495780    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495783    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495785    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495788    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495790    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495794    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495798    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495801    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495804    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495806    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495808    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495811    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495814    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495816    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495819    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495828    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495831    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495834    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495837    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:00.499580 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495839    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495842    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495845    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495848    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495850    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495853    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495856    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495858    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495861    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495864    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495867    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495871    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495875    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495877    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495880    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495883    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495886    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495889    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495891    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:00.500078 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495894    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495897    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495899    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495902    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495904    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495907    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495909    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495912    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495914    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495917    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495925    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495928    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495931    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495933    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495936    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495938    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495941    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495943    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495946    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495948    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:00.500571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495951    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495953    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495955    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495958    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495960    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495963    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495965    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495968    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495970    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495974    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495976    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495981    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495984    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495986    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495988    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495991    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495993    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495996    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.495998    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.496000    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:00.501065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.496003    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.496005    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.496008    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.496016    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.496951    2581 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.496961    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.496966    2581 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.496971    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.496975    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.496979    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.496983    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.496987    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.496990    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.496993    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.496996    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497000    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497003    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497006    2581 flags.go:64] FLAG: --cgroup-root=""
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497009    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497012    2581 flags.go:64] FLAG: --client-ca-file=""
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497015    2581 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497019    2581 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497027    2581 flags.go:64] FLAG: --cluster-domain=""
Oct 30 18:36:00.501571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497030    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497034    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497037    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497040    2581 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497044    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497047    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497051    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497054    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497057    2581 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497060    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497063    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497066    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497069    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497079    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497082    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497085    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497088    2581 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497091    2581 flags.go:64] FLAG: --enable-server="true"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497094    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497101    2581 flags.go:64] FLAG: --event-burst="100"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497104    2581 flags.go:64] FLAG: --event-qps="50"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497107    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497110    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497113    2581 flags.go:64] FLAG: --eviction-hard=""
Oct 30 18:36:00.502134 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497116    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497119    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497122    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497125    2581 flags.go:64] FLAG: --eviction-soft=""
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497128    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497131    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497134    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497137    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497140    2581 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497143    2581 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497146    2581 flags.go:64] FLAG: --feature-gates=""
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497150    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497153    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497156    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497159    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497162    2581 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497165    2581 flags.go:64] FLAG: --help="false"
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497168    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497172    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497175    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497178    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497182    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497191    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 18:36:00.502731 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497194    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497197    2581 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497200    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497202    2581 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497205    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497208    2581 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497211    2581 flags.go:64] FLAG: --kube-reserved=""
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497214    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497216    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497220    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497222    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497225    2581 flags.go:64] FLAG: --lock-file=""
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497228    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497231    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497234    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497242    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497245    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497248    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497251    2581 flags.go:64] FLAG: --logging-format="text"
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497254    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497257    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497260    2581 flags.go:64] FLAG: --manifest-url=""
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497263    2581 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497268    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497273    2581 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 18:36:00.503334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497276    2581 flags.go:64] FLAG: --max-pods="110"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497279    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497282    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497285    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497288    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497291    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497294    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497303    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497310    2581 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497313    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497316    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497319    2581 flags.go:64] FLAG: --pod-cidr=""
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497322    2581 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497325    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497328    2581 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497331    2581 flags.go:64] FLAG: --port="10250"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497334    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497337    2581 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-06aea14225dec2643"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497340    2581 flags.go:64] FLAG: --qos-reserved=""
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497348    2581 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497352    2581 flags.go:64] FLAG: --register-node="true"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497354    2581 flags.go:64] FLAG: --register-with-taints=""
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497358    2581 flags.go:64] FLAG: --registry-burst="10"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497361    2581 flags.go:64] FLAG: --registry-qps="5"
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497364    2581 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497367    2581 flags.go:64] FLAG: --reserved-memory=""
Oct 30 18:36:00.503931 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497371    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497374    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497377    2581 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497380    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497383    2581 flags.go:64] FLAG: --runonce="false"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497388    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497391    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497394    2581 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497397    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497400    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497403    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497406    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497409    2581 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497412    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497422    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497425    2581 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497428    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497432    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497435    2581 flags.go:64] FLAG: --system-cgroups=""
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497438    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497443    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497446    2581 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497449    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497458    2581 flags.go:64] FLAG: --tls-min-version=""
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497461    2581 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 18:36:00.504557 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497464    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497467    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497470    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497473    2581 flags.go:64] FLAG: --v="2"
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497477    2581 flags.go:64] FLAG: --version="false"
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497483    2581 flags.go:64] FLAG: --vmodule=""
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497486    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497490    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497669    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497674    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497676    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497679    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497683    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497686    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497689    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497691    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497694    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497697    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497700    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497702    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497705    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:00.505183 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497708    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497711    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497713    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497716    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497718    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497721    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497724    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497726    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497729    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497731    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497734    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497736    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497739    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497742    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497744    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497747    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497751    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497754    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497757    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:00.505710 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497759    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497762    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497764    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497767    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497771    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497773    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497776    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497778    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497781    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497783    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497786    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497789    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497791    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497794    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497797    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497800    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497803    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497805    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497808    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497810    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:00.506434 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497813    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497816    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497820    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497823    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497826    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497829    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497831    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497834    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497836    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497841    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497845    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497849    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497851    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497854    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497856    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497859    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497863    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497866    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497868    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:00.507065 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497871    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497873    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497876    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497879    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497882    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497884    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497887    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497890    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497892    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497895    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497898    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497900    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497903    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497906    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497908    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497911    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497914    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497916    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497919    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497921    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:00.507516 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497924    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:00.508002 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497926    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:00.508002 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497930    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:00.508002 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497933    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:00.508002 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497935    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:00.508002 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497938    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:00.508002 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497940    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:00.508002 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497943    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:00.508002 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497946    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:00.508002 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497950    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:00.508002 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497952    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:00.508002 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497955    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:00.508002 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497957    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:00.508002 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497960    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:00.508002 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.497970    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:00.508002 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.497998    2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:36:00.508002 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498032    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498036    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498040    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498043    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498046    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498049    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498053    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498055    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498058    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498061    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498064    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498066    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498069    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498072    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498075    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498077    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498080    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498082    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498086    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498089    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:00.508374 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498091    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498094    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498096    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498099    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498101    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498104    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498107    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498110    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498113    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498115    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498118    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498121    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498124    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498126    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498129    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498131    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498134    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498137    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498140    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:00.508911 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498143    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498146    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498148    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498151    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498153    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498156    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498158    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498161    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498163    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498166    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498168    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498172    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498175    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498178    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498180    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498183    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498185    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498188    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498190    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:00.509354 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498194    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498197    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498200    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498202    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498205    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498207    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498210    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498213    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498215    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498218    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498221    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498223    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498226    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498229    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498232    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498235    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498237    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498240    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498242    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:00.509815 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498246    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498249    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498252    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498255    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498258    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498262    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498264    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498267    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498271    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498275    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498278    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498281    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498284    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498287    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498290    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498292    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498295    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498298    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498300    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:00.510255 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498303    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498306    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498309    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498311    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498314    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498317    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498320    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498322    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498325    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.498328    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.504961    2581 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.504977    2581 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.505583    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.505589    2581 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507280    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507453    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507473    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507477    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507481    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507484    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:00.510712 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507488    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507490    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507493    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507496    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507499    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507502    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507505    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507507    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507510    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507513    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507516    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507519    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507521    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507524    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507527    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507531    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507534    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507536    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507540    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507544    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:00.511216 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507546    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507549    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507552    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507556    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507558    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507561    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507564    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507567    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507570    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507573    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507576    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507578    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507581    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507584    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507587    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507589    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507592    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507595    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507597    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507613    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:00.511688 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507617    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507619    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507622    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507625    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507627    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507631    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507633    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507636    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507638    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507641    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507644    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507647    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507650    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507652    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507655    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507658    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507660    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507663    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507665    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507668    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:00.512163 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507671    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507673    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507676    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507678    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507681    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507686    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507690    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507693    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507695    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507698    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507700    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507703    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507706    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507708    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507712    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507716    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507719    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507723    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507726    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:00.512628 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507729    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507731    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507734    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507736    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507739    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507741    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507745    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507747    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507750    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507753    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507755    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507758    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507760    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507763    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507766    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507768    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507771    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507774    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507776    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507779    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:00.513069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507781    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:00.513547 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.507814    2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:36:00.513547 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507967    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:00.513547 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507972    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:00.513547 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507975    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:00.513547 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507978    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:00.513547 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507980    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:00.513547 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507983    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:00.513547 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507986    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:00.513547 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507989    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:00.513547 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507992    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:00.513547 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507995    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:00.513547 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.507998    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:00.513547 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508000    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:00.513547 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508003    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:00.513547 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508005    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:00.513547 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508008    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508011    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508014    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508017    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508019    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508022    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508025    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508027    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508030    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508033    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508035    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508038    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508041    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508044    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508046    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508049    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508052    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508054    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508057    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:00.513937 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508059    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508062    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508064    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508067    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508069    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508072    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508074    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508077    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508080    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508083    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508086    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508088    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508091    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508094    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508096    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508099    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508101    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508103    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508106    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:00.514388 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508109    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508111    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508114    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508116    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508119    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508122    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508126    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508130    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508133    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508136    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508138    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508141    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508143    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508147    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508150    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508153    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508156    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508158    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508161    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:00.514902 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508163    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508166    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508169    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508171    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508174    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508177    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508179    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508182    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508184    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508187    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508189    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508192    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508195    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508198    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508200    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508203    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508206    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508208    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508211    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508213    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:00.515335 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508216    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:00.515805 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508218    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:00.515805 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508221    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:00.515805 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508223    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:00.515805 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508226    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:00.515805 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508228    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:00.515805 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508231    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:00.515805 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508233    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:00.515805 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508236    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:00.515805 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508238    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:00.515805 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508241    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:00.515805 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508243    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:00.515805 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508246    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:00.515805 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508248    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:00.515805 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508251    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:00.515805 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.508274    2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:36:00.515805 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508304    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508309    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508313    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508316    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508318    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508321    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508323    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508326    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508329    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508331    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508334    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508336    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508339    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508342    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508344    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508347    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508349    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508352    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508354    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:00.516177 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508358    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508361    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508364    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508367    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508369    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508372    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508374    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508378    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508382    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508384    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508387    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508390    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508392    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508395    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508397    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508401    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508403    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508406    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508409    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:00.516622 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508411    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508414    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508416    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508419    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508424    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508427    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508430    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508432    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508435    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508438    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508440    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508443    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508445    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508447    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508450    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508452    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508455    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508457    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508460    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508462    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:00.517069 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508465    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508467    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508470    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508472    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508475    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508477    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508480    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508484    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508487    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508490    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508493    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508495    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508498    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508501    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508503    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508506    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508509    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508512    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508514    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508517    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:00.517546 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508519    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508522    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508524    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508527    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508529    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508532    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508534    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508537    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508540    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508542    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508545    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508547    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508551    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508553    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508556    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508558    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508561    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508563    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508566    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:00.518031 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508570    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:00.518474 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508573    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:00.518474 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508575    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:00.518474 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508578    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:00.518474 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508581    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:00.518474 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508583    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:00.518474 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508586    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:00.518474 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508588    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:00.518474 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:00.508591    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:00.518474 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.508663    2581 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 18:36:00.518474 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.510766    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 18:36:00.518474 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.511746    2581 server.go:1014] "Starting client certificate rotation"
Oct 30 18:36:00.518474 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.511839    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:36:00.518474 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.512593    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:36:00.518474 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.513712    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:36:00.518474 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.514809    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:36:00.526579 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.526560    2581 log.go:25] "Validated CRI v1 runtime API"
Oct 30 18:36:00.529698 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.529682    2581 log.go:25] "Validated CRI v1 image API"
Oct 30 18:36:00.530499 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.530476    2581 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 18:36:00.534683 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.534666    2581 fs.go:135] Filesystem UUIDs: map[6e6dd817-e5ce-4d37-8e08-f6c04abfe3d9:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 80bc89d5-3804-4b53-8857-1554ffa4328d:/dev/nvme0n1p3]
Oct 30 18:36:00.534732 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.534682    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 18:36:00.540371 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.540257    2581 manager.go:217] Machine: {Timestamp:2026-10-30 18:36:00.538940027 +0000 UTC m=+0.319424668 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098026 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec204d1e3a779c06e5f9875099646a10 SystemUUID:ec204d1e-3a77-9c06-e5f9-875099646a10 BootID:f634fc99-5ea3-4a46-bf49-221d2d807091 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:eb:79:f9:57 Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:eb:79:f9:57 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:f9:ae:9b:d0:aa Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 18:36:00.540371 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.540370    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 18:36:00.540483 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.540472    2581 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 18:36:00.544456 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.544266    2581 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 18:36:00.544586 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.544459    2581 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-122.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 18:36:00.545146 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.545136    2581 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 18:36:00.545188 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.545148    2581 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 18:36:00.545188 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.545175    2581 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:36:00.545239 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.545188    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:36:00.545932 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.545923    2581 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 18:36:00.548347 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.548336    2581 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 18:36:00.548462 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.548453    2581 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 18:36:00.549106 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.549097    2581 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 18:36:00.549166 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.549109    2581 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 18:36:00.549166 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.549123    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 18:36:00.549166 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.549131    2581 kubelet.go:405] "Adding apiserver pod source"
Oct 30 18:36:00.549166 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.549139    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 18:36:00.549457 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.549446    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:36:00.551597 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.551576    2581 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 18:36:00.555833 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.555818    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:36:00.556610 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.556586    2581 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 18:36:00.556650 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.556630    2581 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 18:36:00.556797 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.556781    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 18:36:00.556826 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.556811    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 18:36:00.556859 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.556827    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 18:36:00.556859 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.556837    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 18:36:00.556859 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.556849    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 18:36:00.556941 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.556862    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 18:36:00.557544 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.557534    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 18:36:00.557576 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.557545    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 18:36:00.557576 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.557552    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 18:36:00.557576 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.557558    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 18:36:00.557576 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.557567    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 18:36:00.557576 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.557577    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 18:36:00.558863 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.558853    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 18:36:00.558904 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.558865    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 18:36:00.562486 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.562471    2581 server.go:1263] "Started kubelet"
Oct 30 18:36:00.562614 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.562519    2581 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 18:36:00.562771 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.562644    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 18:36:00.562866 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.562855    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 18:36:00.563365 ip-10-0-140-122 systemd[1]: Started Kubernetes Kubelet.
Oct 30 18:36:00.563915 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.563892    2581 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 18:36:00.564466 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.564454    2581 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 18:36:00.569550 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:00.569498    2581 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 18:36:00.569906 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.569871    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 18:36:00.570331 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.570302    2581 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 18:36:00.570487 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:00.570465    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-122.us-west-2.compute.internal\" not found"
Oct 30 18:36:00.571072 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.571029    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 18:36:00.571183 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.571031    2581 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 18:36:00.571183 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.571181    2581 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 18:36:00.571316 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.571183    2581 factory.go:55] Registering systemd factory
Oct 30 18:36:00.571316 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.571204    2581 factory.go:223] Registration of the systemd container factory successfully
Oct 30 18:36:00.571403 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.571324    2581 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 18:36:00.571403 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.571336    2581 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 18:36:00.571502 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.571491    2581 factory.go:153] Registering CRI-O factory
Oct 30 18:36:00.571546 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.571508    2581 factory.go:223] Registration of the crio container factory successfully
Oct 30 18:36:00.571593 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.571564    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 18:36:00.571593 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.571581    2581 factory.go:103] Registering Raw factory
Oct 30 18:36:00.571593 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.571592    2581 manager.go:1199] Started watching for new ooms in manager
Oct 30 18:36:00.572438 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.572423    2581 manager.go:319] Starting recovery of all containers
Oct 30 18:36:00.582847 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.582803    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 18:36:00.583248 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.583233    2581 manager.go:324] Recovery completed
Oct 30 18:36:00.587218 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.587192    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:00.589424 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.588876    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:00.589424 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.588901    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:00.589424 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.588912    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:36:00.589424 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.589348    2581 cpu_manager.go:226] "Starting" policy="none"
Oct 30 18:36:00.589424 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.589357    2581 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 18:36:00.589424 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.589373    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:00.589725 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.589375    2581 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 18:36:00.590162 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.590147    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:00.590246 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.590177    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:00.590246 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.590191    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:36:00.670647 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:00.670624    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-122.us-west-2.compute.internal\" not found"
Oct 30 18:36:00.697778 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:00.697750    2581 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-122.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 18:36:00.697884 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.697758    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-122.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 18:36:00.698799 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:00.697763    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-122.us-west-2.compute.internal.18e3627bd1b5cb8b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-122.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-122.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 18:36:00.562441099 +0000 UTC m=+0.342925739,LastTimestamp:2026-10-30 18:36:00.562441099 +0000 UTC m=+0.342925739,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-122.us-west-2.compute.internal,}"
Oct 30 18:36:00.726177 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.726154    2581 policy_none.go:50] "Start"
Oct 30 18:36:00.726177 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.726177    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 18:36:00.726282 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.726197    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 18:36:00.727182 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.727170    2581 policy_none.go:44] "Start"
Oct 30 18:36:00.754384 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:00.754352    2581 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 18:36:00.754744 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:00.754348    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-122.us-west-2.compute.internal.18e3627bd3495826  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-122.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-122.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-122.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 18:36:00.588888102 +0000 UTC m=+0.369372743,LastTimestamp:2026-10-30 18:36:00.588888102 +0000 UTC m=+0.369372743,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-122.us-west-2.compute.internal,}"
Oct 30 18:36:00.754854 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:00.754767    2581 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 18:36:00.754916 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:00.754896    2581 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-122.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 18:36:00.769200 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.769164    2581 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 18:36:00.771390 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:00.771371    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-122.us-west-2.compute.internal\" not found"
Oct 30 18:36:00.771492 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.771468    2581 manager.go:341] "Starting Device Plugin manager"
Oct 30 18:36:00.772226 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:00.772209    2581 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 18:36:00.772226 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.772222    2581 server.go:87] "Starting device plugin registration server"
Oct 30 18:36:00.772451 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.772438    2581 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 18:36:00.772518 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.772451    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 18:36:00.772574 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.772536    2581 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 18:36:00.772646 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.772624    2581 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 18:36:00.772646 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.772631    2581 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 18:36:00.773135 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:00.773115    2581 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 18:36:00.773216 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:00.773160    2581 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-122.us-west-2.compute.internal\" not found"
Oct 30 18:36:00.776843 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.776829    2581 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 18:36:00.776895 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.776882    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cnv64"
Oct 30 18:36:00.804796 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.804774    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 18:36:00.804796 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.804795    2581 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 18:36:00.804937 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.804813    2581 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 18:36:00.804937 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:00.804842    2581 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 18:36:00.855284 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.855266    2581 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:00.873528 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.873507    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:00.876329 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.876315    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:00.876388 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.876343    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:00.876388 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.876355    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:36:00.876388 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.876378    2581 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:00.905919 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.905895    2581 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal"]
Oct 30 18:36:00.905997 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.905956    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:00.906929 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.906914    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:00.907016 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.906943    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:00.907016 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.906953    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:36:00.909252 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.909241    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:00.909398 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.909383    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:00.909433 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.909417    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:00.910099 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.910081    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:00.910209 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.910112    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:00.910209 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.910128    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:36:00.910209 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.910081    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:00.910209 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.910189    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:00.910209 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.910205    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:36:00.912367 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.912351    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:00.912451 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.912381    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:00.913217 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.913200    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:00.913310 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.913229    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:00.913310 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.913241    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:36:00.930092 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.930074    2581 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:00.930166 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:00.930097    2581 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-122.us-west-2.compute.internal\": node \"ip-10-0-140-122.us-west-2.compute.internal\" not found"
Oct 30 18:36:00.939725 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:00.939708    2581 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-122.us-west-2.compute.internal\" not found" node="ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:00.942112 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:00.942096    2581 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-122.us-west-2.compute.internal\" not found" node="ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:00.972531 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.972504    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d48fbbc6436568b80ec3382fdeed930-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal\" (UID: \"7d48fbbc6436568b80ec3382fdeed930\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:00.972662 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.972537    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7018e001fd99c1f4ae76683b64500513-config\") pod \"kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal\" (UID: \"7018e001fd99c1f4ae76683b64500513\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:00.972662 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:00.972555    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7d48fbbc6436568b80ec3382fdeed930-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal\" (UID: \"7d48fbbc6436568b80ec3382fdeed930\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:01.041911 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:01.041833    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-122.us-west-2.compute.internal\" not found"
Oct 30 18:36:01.073153 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.073119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d48fbbc6436568b80ec3382fdeed930-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal\" (UID: \"7d48fbbc6436568b80ec3382fdeed930\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:01.073153 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.073152    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7018e001fd99c1f4ae76683b64500513-config\") pod \"kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal\" (UID: \"7018e001fd99c1f4ae76683b64500513\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:01.073361 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.073174    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7d48fbbc6436568b80ec3382fdeed930-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal\" (UID: \"7d48fbbc6436568b80ec3382fdeed930\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:01.073361 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.073233    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7d48fbbc6436568b80ec3382fdeed930-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal\" (UID: \"7d48fbbc6436568b80ec3382fdeed930\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:01.073361 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.073247    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7018e001fd99c1f4ae76683b64500513-config\") pod \"kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal\" (UID: \"7018e001fd99c1f4ae76683b64500513\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:01.073361 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.073247    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d48fbbc6436568b80ec3382fdeed930-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal\" (UID: \"7d48fbbc6436568b80ec3382fdeed930\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:01.142351 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:01.142311    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-122.us-west-2.compute.internal\" not found"
Oct 30 18:36:01.243135 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:01.243107    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-122.us-west-2.compute.internal\" not found"
Oct 30 18:36:01.243233 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.243143    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:01.245320 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.245293    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:01.344370 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:01.344152    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-122.us-west-2.compute.internal\" not found"
Oct 30 18:36:01.444391 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:01.444355    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-122.us-west-2.compute.internal\" not found"
Oct 30 18:36:01.511884 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.511847    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 18:36:01.512363 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.512065    2581 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 18:36:01.512363 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:01.512045    2581 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a511c06f6c4c24b5eb9148195e6ce288-46029cde054419ce.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-140-122.us-west-2.compute.internal.18e3627bd349c477\": read tcp 10.0.140.122:54072->77.112.254.244:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-140-122.us-west-2.compute.internal.18e3627bd349c477  default   4990 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-122.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-122.us-west-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-122.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 18:36:00 +0000 UTC,LastTimestamp:2026-10-30 18:36:00.906964051 +0000 UTC m=+0.687448691,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-122.us-west-2.compute.internal,}"
Oct 30 18:36:01.545814 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:01.545786    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-122.us-west-2.compute.internal\" not found"
Oct 30 18:36:01.648722 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:01.648639    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-122.us-west-2.compute.internal\" not found"
Oct 30 18:36:01.741840 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.741812    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:01.742849 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.742830    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:01.742936 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.742865    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:01.742936 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.742877    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:36:01.748844 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:01.748829    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-122.us-west-2.compute.internal\" not found"
Oct 30 18:36:01.781467 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.781420    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:31:01 +0000 UTC" deadline="2028-05-03 07:27:25.232946746 +0000 UTC"
Oct 30 18:36:01.781467 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.781462    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13212h51m23.451489251s"
Oct 30 18:36:01.850000 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:01.849975    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-122.us-west-2.compute.internal\" not found"
Oct 30 18:36:01.859394 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.859376    2581 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:01.906406 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.906349    2581 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:01.957375 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.957350    2581 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:01.971012 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:01.970986    2581 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:02.031013 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.030986    2581 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:36:02.032145 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.032127    2581 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:02.091718 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.091699    2581 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:36:02.453859 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:02.453814    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7018e001fd99c1f4ae76683b64500513.slice/crio-9963f1102736ee5cf0c733cf42c99fd2f2cec60d380ed11e1d351fed5622cbce WatchSource:0}: Error finding container 9963f1102736ee5cf0c733cf42c99fd2f2cec60d380ed11e1d351fed5622cbce: Status 404 returned error can't find the container with id 9963f1102736ee5cf0c733cf42c99fd2f2cec60d380ed11e1d351fed5622cbce
Oct 30 18:36:02.458419 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:02.458391    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d48fbbc6436568b80ec3382fdeed930.slice/crio-3da38b7898a74e5db9cfbea383d4ba1963ac2528558e42bac58423fd58d14ac0 WatchSource:0}: Error finding container 3da38b7898a74e5db9cfbea383d4ba1963ac2528558e42bac58423fd58d14ac0: Status 404 returned error can't find the container with id 3da38b7898a74e5db9cfbea383d4ba1963ac2528558e42bac58423fd58d14ac0
Oct 30 18:36:02.460636 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.460617    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:36:02.550673 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.550621    2581 apiserver.go:52] "Watching apiserver"
Oct 30 18:36:02.572092 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.572064    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 18:36:02.654940 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.654854    2581 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 18:36:02.655161 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.655144    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-qkvdd","openshift-multus/network-metrics-daemon-7wdcf","openshift-network-diagnostics/network-check-target-qk8z2","openshift-dns/node-resolver-nmmwh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal","openshift-multus/multus-additional-cni-plugins-cxhd5","kube-system/global-pull-secret-syncer-b4fj5","kube-system/konnectivity-agent-r52tw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz","openshift-cluster-node-tuning-operator/tuned-78mjc","openshift-network-operator/iptables-alerter-nssbw","kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal","openshift-multus/multus-54jpr","openshift-ovn-kubernetes/ovnkube-node-nxnnz"]
Oct 30 18:36:02.659141 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.659123    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:02.659230 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:02.659211    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4fj5" podUID="f51455e1-dbba-4186-adbf-60f44c9edbe5"
Oct 30 18:36:02.661219 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.661205    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nssbw"
Oct 30 18:36:02.663484 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.663448    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:02.663610 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:02.663528    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qk8z2" podUID="f3e9afad-06d1-42eb-b463-5dc5932cd45d"
Oct 30 18:36:02.665879 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.665861    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nmmwh"
Oct 30 18:36:02.668179 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.668158    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.670489 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.670463    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.672787 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.672766    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.675280 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.675264    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.677647 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.677631    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:02.677715 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:02.677698    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wdcf" podUID="92a21826-d145-4b9b-9897-aaf21f63a67b"
Oct 30 18:36:02.680198 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.680177    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.680926 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.680909    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdsjl\" (UniqueName: \"kubernetes.io/projected/f7894bc9-1a77-42e3-a59b-926729c93dc9-kube-api-access-bdsjl\") pod \"iptables-alerter-nssbw\" (UID: \"f7894bc9-1a77-42e3-a59b-926729c93dc9\") " pod="openshift-network-operator/iptables-alerter-nssbw"
Oct 30 18:36:02.681023 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.680938    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhn6w\" (UniqueName: \"kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w\") pod \"network-check-target-qk8z2\" (UID: \"f3e9afad-06d1-42eb-b463-5dc5932cd45d\") " pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:02.681023 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.680981    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-log-socket\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.681023 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681001    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-cni-bin\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.681151 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681047    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-cni-netd\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.681151 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681070    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/114fbcb8-1004-410d-98bc-dc60f8fcae3c-ovnkube-script-lib\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.681151 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681104    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-etc-selinux\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.681151 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681143    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:02.681350 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681164    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-var-lib-openvswitch\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.681350 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681209    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-etc-openvswitch\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.681350 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681262    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-run-ovn-kubernetes\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.681350 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681308    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pphxn\" (UniqueName: \"kubernetes.io/projected/114fbcb8-1004-410d-98bc-dc60f8fcae3c-kube-api-access-pphxn\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.681350 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681343    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab0116fc-0400-4f7d-be91-27804b24c8f4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.681538 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681367    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-sys-fs\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.681538 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681403    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-run-systemd\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.681538 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681433    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.681538 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681452    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/114fbcb8-1004-410d-98bc-dc60f8fcae3c-ovn-node-metrics-cert\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.681538 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681469    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab0116fc-0400-4f7d-be91-27804b24c8f4-system-cni-dir\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.681538 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681499    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab0116fc-0400-4f7d-be91-27804b24c8f4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.681538 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681531    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgj6t\" (UniqueName: \"kubernetes.io/projected/ab0116fc-0400-4f7d-be91-27804b24c8f4-kube-api-access-tgj6t\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.681839 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681587    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d7ebad18-eccb-47c9-b259-67c6d6b84d0f-hosts-file\") pod \"node-resolver-nmmwh\" (UID: \"d7ebad18-eccb-47c9-b259-67c6d6b84d0f\") " pod="openshift-dns/node-resolver-nmmwh"
Oct 30 18:36:02.681839 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681640    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f51455e1-dbba-4186-adbf-60f44c9edbe5-dbus\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:02.681839 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681684    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7894bc9-1a77-42e3-a59b-926729c93dc9-iptables-alerter-script\") pod \"iptables-alerter-nssbw\" (UID: \"f7894bc9-1a77-42e3-a59b-926729c93dc9\") " pod="openshift-network-operator/iptables-alerter-nssbw"
Oct 30 18:36:02.681839 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681728    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/114fbcb8-1004-410d-98bc-dc60f8fcae3c-ovnkube-config\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.681839 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681757    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-device-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.681839 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681781    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8clvh\" (UniqueName: \"kubernetes.io/projected/90c6593d-b40a-43ae-abc0-ce015e48c289-kube-api-access-8clvh\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.681839 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681814    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7ebad18-eccb-47c9-b259-67c6d6b84d0f-tmp-dir\") pod \"node-resolver-nmmwh\" (UID: \"d7ebad18-eccb-47c9-b259-67c6d6b84d0f\") " pod="openshift-dns/node-resolver-nmmwh"
Oct 30 18:36:02.682073 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681855    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f51455e1-dbba-4186-adbf-60f44c9edbe5-kubelet-config\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:02.682073 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681885    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-slash\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.682073 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681920    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-socket-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.682073 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681937    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7894bc9-1a77-42e3-a59b-926729c93dc9-host-slash\") pod \"iptables-alerter-nssbw\" (UID: \"f7894bc9-1a77-42e3-a59b-926729c93dc9\") " pod="openshift-network-operator/iptables-alerter-nssbw"
Oct 30 18:36:02.682073 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.681986    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-run-netns\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.682073 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.682019    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-node-log\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.682073 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.682038    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab0116fc-0400-4f7d-be91-27804b24c8f4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.682073 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.682067    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hwhq\" (UniqueName: \"kubernetes.io/projected/d7ebad18-eccb-47c9-b259-67c6d6b84d0f-kube-api-access-2hwhq\") pod \"node-resolver-nmmwh\" (UID: \"d7ebad18-eccb-47c9-b259-67c6d6b84d0f\") " pod="openshift-dns/node-resolver-nmmwh"
Oct 30 18:36:02.682339 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.682098    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-kubelet\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.682339 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.682115    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-systemd-units\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.682339 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.682137    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/114fbcb8-1004-410d-98bc-dc60f8fcae3c-env-overrides\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.682339 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.682167    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab0116fc-0400-4f7d-be91-27804b24c8f4-cnibin\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.682339 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.682187    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab0116fc-0400-4f7d-be91-27804b24c8f4-cni-binary-copy\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.682339 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.682204    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-registration-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.682339 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.682226    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-run-openvswitch\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.682339 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.682253    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-run-ovn\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.682339 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.682277    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.682784 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.682766    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qkvdd"
Oct 30 18:36:02.685249 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.685234    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r52tw"
Oct 30 18:36:02.715695 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.715657    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:02.715901 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.715882    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 18:36:02.715998 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.715980    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 18:36:02.716057 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.716037    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dnpxx\""
Oct 30 18:36:02.719833 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.719758    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 18:36:02.719833 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.719807    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 18:36:02.719968 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.719851    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vpn2j\""
Oct 30 18:36:02.720409 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.720391    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 18:36:02.720492 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.720454    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 18:36:02.720629 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.720606    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 18:36:02.756139 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.756103    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 18:36:02.756139 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.756117    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 18:36:02.756350 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.756336    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 18:36:02.756458 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.756443    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 18:36:02.756501 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.756455    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 18:36:02.756501 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.756461    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 18:36:02.756598 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.756533    2581 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:02.756598 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.756568    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 18:36:02.756598 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.756588    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 18:36:02.756758 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.756603    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 18:36:02.756758 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.756621    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 18:36:02.756758 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.756676    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 18:36:02.756973 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.756959    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j6hdp\""
Oct 30 18:36:02.757067 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.757051    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tlfn7\""
Oct 30 18:36:02.757109 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.757070    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 18:36:02.757109 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.757090    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-7kzh2\""
Oct 30 18:36:02.765051 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.765036    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:02.765415 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.765394    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 18:36:02.765675 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.765653    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vd777\""
Oct 30 18:36:02.765771 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.765680    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 18:36:02.765771 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.765738    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 18:36:02.765860 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.765774    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-5mz5g\""
Oct 30 18:36:02.765860 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.765781    2581 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 18:36:02.765860 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.765786    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 18:36:02.765860 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.765813    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-h4btl\""
Oct 30 18:36:02.765860 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.765832    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-g295c"
Oct 30 18:36:02.765860 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.765847    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 18:36:02.766068 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.765878    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 18:36:02.766068 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.765881    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m6qzp\""
Oct 30 18:36:02.772616 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.772600    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 18:36:02.782814 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.782794    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7ebad18-eccb-47c9-b259-67c6d6b84d0f-tmp-dir\") pod \"node-resolver-nmmwh\" (UID: \"d7ebad18-eccb-47c9-b259-67c6d6b84d0f\") " pod="openshift-dns/node-resolver-nmmwh"
Oct 30 18:36:02.782866 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.782823    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-socket-dir-parent\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.782866 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.782840    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnjp7\" (UniqueName: \"kubernetes.io/projected/a517bba6-536f-4dac-ba8b-2e71964ef8c5-kube-api-access-vnjp7\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.782866 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.782859    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-var-lib-kubelet\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.783017 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.782875    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-slash\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.783017 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.782919    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-socket-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.783017 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.782982    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2fqp\" (UniqueName: \"kubernetes.io/projected/92a21826-d145-4b9b-9897-aaf21f63a67b-kube-api-access-c2fqp\") pod \"network-metrics-daemon-7wdcf\" (UID: \"92a21826-d145-4b9b-9897-aaf21f63a67b\") " pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:02.783017 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.782999    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhx9h\" (UniqueName: \"kubernetes.io/projected/d93ec858-ae33-473a-ad4b-18b88664094c-kube-api-access-dhx9h\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.783017 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783016    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npfv7\" (UniqueName: \"kubernetes.io/projected/4a6021d6-6317-473f-97bc-ea023261d90b-kube-api-access-npfv7\") pod \"node-ca-qkvdd\" (UID: \"4a6021d6-6317-473f-97bc-ea023261d90b\") " pod="openshift-image-registry/node-ca-qkvdd"
Oct 30 18:36:02.783279 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783046    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-node-log\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.783279 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783078    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-slash\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.783279 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783088    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hwhq\" (UniqueName: \"kubernetes.io/projected/d7ebad18-eccb-47c9-b259-67c6d6b84d0f-kube-api-access-2hwhq\") pod \"node-resolver-nmmwh\" (UID: \"d7ebad18-eccb-47c9-b259-67c6d6b84d0f\") " pod="openshift-dns/node-resolver-nmmwh"
Oct 30 18:36:02.783279 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783113    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-node-log\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.783279 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783135    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-socket-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.783279 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783157    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-hostroot\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.783279 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783207    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7ebad18-eccb-47c9-b259-67c6d6b84d0f-tmp-dir\") pod \"node-resolver-nmmwh\" (UID: \"d7ebad18-eccb-47c9-b259-67c6d6b84d0f\") " pod="openshift-dns/node-resolver-nmmwh"
Oct 30 18:36:02.783279 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783264    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d93ec858-ae33-473a-ad4b-18b88664094c-etc-tuned\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.783587 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783302    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-registration-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.783587 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783340    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-var-lib-cni-bin\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.783587 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783363    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-registration-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.783587 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783391    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.783587 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783407    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-run\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.783587 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783433    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhn6w\" (UniqueName: \"kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w\") pod \"network-check-target-qk8z2\" (UID: \"f3e9afad-06d1-42eb-b463-5dc5932cd45d\") " pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:02.783587 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783474    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.783587 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783483    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-log-socket\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.783587 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783517    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-etc-selinux\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.783587 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783554    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-sys-fs\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.783587 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783582    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-log-socket\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.784056 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783634    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-cnibin\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.784056 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783653    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-sys-fs\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.784056 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783686    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d93ec858-ae33-473a-ad4b-18b88664094c-tmp\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.784056 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783689    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-etc-selinux\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.784056 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783730    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-var-lib-openvswitch\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.784056 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783761    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pphxn\" (UniqueName: \"kubernetes.io/projected/114fbcb8-1004-410d-98bc-dc60f8fcae3c-kube-api-access-pphxn\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.784056 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783783    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-var-lib-openvswitch\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.784056 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783790    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-run-multus-certs\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.784056 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783819    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-sysctl-conf\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.784056 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783855    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a6021d6-6317-473f-97bc-ea023261d90b-host\") pod \"node-ca-qkvdd\" (UID: \"4a6021d6-6317-473f-97bc-ea023261d90b\") " pod="openshift-image-registry/node-ca-qkvdd"
Oct 30 18:36:02.784056 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783876    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4a6021d6-6317-473f-97bc-ea023261d90b-serviceca\") pod \"node-ca-qkvdd\" (UID: \"4a6021d6-6317-473f-97bc-ea023261d90b\") " pod="openshift-image-registry/node-ca-qkvdd"
Oct 30 18:36:02.784056 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783892    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-run-netns\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.784056 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783947    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab0116fc-0400-4f7d-be91-27804b24c8f4-system-cni-dir\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.784056 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783974    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgj6t\" (UniqueName: \"kubernetes.io/projected/ab0116fc-0400-4f7d-be91-27804b24c8f4-kube-api-access-tgj6t\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.784056 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.783991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7894bc9-1a77-42e3-a59b-926729c93dc9-iptables-alerter-script\") pod \"iptables-alerter-nssbw\" (UID: \"f7894bc9-1a77-42e3-a59b-926729c93dc9\") " pod="openshift-network-operator/iptables-alerter-nssbw"
Oct 30 18:36:02.784056 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784039    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab0116fc-0400-4f7d-be91-27804b24c8f4-system-cni-dir\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.784056 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784046    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-device-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.784732 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784100    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-device-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.784732 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784119    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-daemon-config\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.784732 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-sysconfig\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.784732 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784153    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-kubernetes\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.784732 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784168    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-sys\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.784732 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f51455e1-dbba-4186-adbf-60f44c9edbe5-kubelet-config\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:02.784732 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784222    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-modprobe-d\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.784732 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784250    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-sysctl-d\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.784732 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784258    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f51455e1-dbba-4186-adbf-60f44c9edbe5-kubelet-config\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:02.784732 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784371    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7894bc9-1a77-42e3-a59b-926729c93dc9-host-slash\") pod \"iptables-alerter-nssbw\" (UID: \"f7894bc9-1a77-42e3-a59b-926729c93dc9\") " pod="openshift-network-operator/iptables-alerter-nssbw"
Oct 30 18:36:02.784732 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784405    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-run-netns\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.784732 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784430    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab0116fc-0400-4f7d-be91-27804b24c8f4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.784732 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784460    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-run-k8s-cni-cncf-io\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.784732 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784461    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7894bc9-1a77-42e3-a59b-926729c93dc9-host-slash\") pod \"iptables-alerter-nssbw\" (UID: \"f7894bc9-1a77-42e3-a59b-926729c93dc9\") " pod="openshift-network-operator/iptables-alerter-nssbw"
Oct 30 18:36:02.784732 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784461    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-run-netns\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.784732 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784512    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-var-lib-cni-multus\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.784732 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784515    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7894bc9-1a77-42e3-a59b-926729c93dc9-iptables-alerter-script\") pod \"iptables-alerter-nssbw\" (UID: \"f7894bc9-1a77-42e3-a59b-926729c93dc9\") " pod="openshift-network-operator/iptables-alerter-nssbw"
Oct 30 18:36:02.785397 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784534    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-kubelet\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.785397 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784556    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-kubelet\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.785397 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784570    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-systemd-units\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.785397 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784660    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/114fbcb8-1004-410d-98bc-dc60f8fcae3c-env-overrides\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.785397 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784671    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-systemd-units\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.785397 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784694    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab0116fc-0400-4f7d-be91-27804b24c8f4-cnibin\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.785397 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784747    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab0116fc-0400-4f7d-be91-27804b24c8f4-cnibin\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.785397 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784784    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab0116fc-0400-4f7d-be91-27804b24c8f4-cni-binary-copy\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.785397 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784804    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a517bba6-536f-4dac-ba8b-2e71964ef8c5-cni-binary-copy\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.785397 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784820    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-host\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.785397 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784840    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-run-openvswitch\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.785397 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784878    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-run-ovn\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.785397 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784898    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs\") pod \"network-metrics-daemon-7wdcf\" (UID: \"92a21826-d145-4b9b-9897-aaf21f63a67b\") " pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:02.785397 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784902    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-run-openvswitch\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.785397 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784913    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-systemd\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.785397 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784947    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-run-ovn\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.785397 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784969    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/299ac5c9-51d1-4903-b7f1-2001aa7b2bcf-agent-certs\") pod \"konnectivity-agent-r52tw\" (UID: \"299ac5c9-51d1-4903-b7f1-2001aa7b2bcf\") " pod="kube-system/konnectivity-agent-r52tw"
Oct 30 18:36:02.786103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.784987    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdsjl\" (UniqueName: \"kubernetes.io/projected/f7894bc9-1a77-42e3-a59b-926729c93dc9-kube-api-access-bdsjl\") pod \"iptables-alerter-nssbw\" (UID: \"f7894bc9-1a77-42e3-a59b-926729c93dc9\") " pod="openshift-network-operator/iptables-alerter-nssbw"
Oct 30 18:36:02.786103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785002    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-cni-bin\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.786103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785015    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab0116fc-0400-4f7d-be91-27804b24c8f4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.786103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785024    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-cni-netd\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.786103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785057    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/114fbcb8-1004-410d-98bc-dc60f8fcae3c-ovnkube-script-lib\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.786103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785088    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/114fbcb8-1004-410d-98bc-dc60f8fcae3c-env-overrides\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.786103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785094    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-cni-bin\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.786103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785131    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8clvh\" (UniqueName: \"kubernetes.io/projected/90c6593d-b40a-43ae-abc0-ce015e48c289-kube-api-access-8clvh\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.786103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785140    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-cni-netd\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.786103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-cni-dir\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.786103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785205    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/299ac5c9-51d1-4903-b7f1-2001aa7b2bcf-konnectivity-ca\") pod \"konnectivity-agent-r52tw\" (UID: \"299ac5c9-51d1-4903-b7f1-2001aa7b2bcf\") " pod="kube-system/konnectivity-agent-r52tw"
Oct 30 18:36:02.786103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785246    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:02.786103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785251    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab0116fc-0400-4f7d-be91-27804b24c8f4-cni-binary-copy\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.786103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785275    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-etc-openvswitch\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.786103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785308    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-run-ovn-kubernetes\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.786103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785345    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-etc-openvswitch\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.786103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785393    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab0116fc-0400-4f7d-be91-27804b24c8f4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.786622 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:02.785407    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:02.786622 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785398    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-run-ovn-kubernetes\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.786622 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785423    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/114fbcb8-1004-410d-98bc-dc60f8fcae3c-ovnkube-script-lib\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.786622 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785433    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-system-cni-dir\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.786622 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:02.785493    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret podName:f51455e1-dbba-4186-adbf-60f44c9edbe5 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:03.28545455 +0000 UTC m=+3.065939201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret") pod "global-pull-secret-syncer-b4fj5" (UID: "f51455e1-dbba-4186-adbf-60f44c9edbe5") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:02.786622 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785516    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-etc-kubernetes\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.786622 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785547    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-run-systemd\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.786622 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785575    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.786622 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785604    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/114fbcb8-1004-410d-98bc-dc60f8fcae3c-ovn-node-metrics-cert\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.786622 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785636    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.786622 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785648    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab0116fc-0400-4f7d-be91-27804b24c8f4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.786622 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785674    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d7ebad18-eccb-47c9-b259-67c6d6b84d0f-hosts-file\") pod \"node-resolver-nmmwh\" (UID: \"d7ebad18-eccb-47c9-b259-67c6d6b84d0f\") " pod="openshift-dns/node-resolver-nmmwh"
Oct 30 18:36:02.786622 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785677    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-run-systemd\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.786622 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785704    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-var-lib-kubelet\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.786622 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785733    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-conf-dir\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.786622 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-lib-modules\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.786622 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785766    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d7ebad18-eccb-47c9-b259-67c6d6b84d0f-hosts-file\") pod \"node-resolver-nmmwh\" (UID: \"d7ebad18-eccb-47c9-b259-67c6d6b84d0f\") " pod="openshift-dns/node-resolver-nmmwh"
Oct 30 18:36:02.787170 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785784    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f51455e1-dbba-4186-adbf-60f44c9edbe5-dbus\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:02.787170 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785829    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/114fbcb8-1004-410d-98bc-dc60f8fcae3c-ovnkube-config\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.787170 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785905    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab0116fc-0400-4f7d-be91-27804b24c8f4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.787170 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785948    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab0116fc-0400-4f7d-be91-27804b24c8f4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.787170 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.785969    2581 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 18:36:02.787170 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.786004    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f51455e1-dbba-4186-adbf-60f44c9edbe5-dbus\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:02.787170 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.786230    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/114fbcb8-1004-410d-98bc-dc60f8fcae3c-ovnkube-config\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.789694 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.789667    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/114fbcb8-1004-410d-98bc-dc60f8fcae3c-ovn-node-metrics-cert\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.808733 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.808682    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal" event={"ID":"7d48fbbc6436568b80ec3382fdeed930","Type":"ContainerStarted","Data":"3da38b7898a74e5db9cfbea383d4ba1963ac2528558e42bac58423fd58d14ac0"}
Oct 30 18:36:02.809698 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.809676    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal" event={"ID":"7018e001fd99c1f4ae76683b64500513","Type":"ContainerStarted","Data":"9963f1102736ee5cf0c733cf42c99fd2f2cec60d380ed11e1d351fed5622cbce"}
Oct 30 18:36:02.839258 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:02.839230    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:36:02.839258 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:02.839256    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:36:02.839391 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:02.839270    2581 projected.go:196] Error preparing data for projected volume kube-api-access-fhn6w for pod openshift-network-diagnostics/network-check-target-qk8z2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:02.839391 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:02.839330    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w podName:f3e9afad-06d1-42eb-b463-5dc5932cd45d nodeName:}" failed. No retries permitted until 2026-10-30 18:36:03.339310813 +0000 UTC m=+3.119795461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fhn6w" (UniqueName: "kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w") pod "network-check-target-qk8z2" (UID: "f3e9afad-06d1-42eb-b463-5dc5932cd45d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:02.841835 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.841810    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hwhq\" (UniqueName: \"kubernetes.io/projected/d7ebad18-eccb-47c9-b259-67c6d6b84d0f-kube-api-access-2hwhq\") pod \"node-resolver-nmmwh\" (UID: \"d7ebad18-eccb-47c9-b259-67c6d6b84d0f\") " pod="openshift-dns/node-resolver-nmmwh"
Oct 30 18:36:02.842276 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.842256    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgj6t\" (UniqueName: \"kubernetes.io/projected/ab0116fc-0400-4f7d-be91-27804b24c8f4-kube-api-access-tgj6t\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:02.842372 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.842347    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pphxn\" (UniqueName: \"kubernetes.io/projected/114fbcb8-1004-410d-98bc-dc60f8fcae3c-kube-api-access-pphxn\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:02.842710 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.842690    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8clvh\" (UniqueName: \"kubernetes.io/projected/90c6593d-b40a-43ae-abc0-ce015e48c289-kube-api-access-8clvh\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:02.842785 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.842708    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdsjl\" (UniqueName: \"kubernetes.io/projected/f7894bc9-1a77-42e3-a59b-926729c93dc9-kube-api-access-bdsjl\") pod \"iptables-alerter-nssbw\" (UID: \"f7894bc9-1a77-42e3-a59b-926729c93dc9\") " pod="openshift-network-operator/iptables-alerter-nssbw"
Oct 30 18:36:02.886826 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.886798    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-cnibin\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.886826 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.886825    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d93ec858-ae33-473a-ad4b-18b88664094c-tmp\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.886980 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.886847    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-run-multus-certs\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.886980 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.886879    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-sysctl-conf\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.886980 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.886896    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a6021d6-6317-473f-97bc-ea023261d90b-host\") pod \"node-ca-qkvdd\" (UID: \"4a6021d6-6317-473f-97bc-ea023261d90b\") " pod="openshift-image-registry/node-ca-qkvdd"
Oct 30 18:36:02.886980 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.886936    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-cnibin\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.886980 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.886936    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-run-multus-certs\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.886980 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.886963    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4a6021d6-6317-473f-97bc-ea023261d90b-serviceca\") pod \"node-ca-qkvdd\" (UID: \"4a6021d6-6317-473f-97bc-ea023261d90b\") " pod="openshift-image-registry/node-ca-qkvdd"
Oct 30 18:36:02.886980 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.886943    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a6021d6-6317-473f-97bc-ea023261d90b-host\") pod \"node-ca-qkvdd\" (UID: \"4a6021d6-6317-473f-97bc-ea023261d90b\") " pod="openshift-image-registry/node-ca-qkvdd"
Oct 30 18:36:02.887336 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.886994    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-run-netns\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.887336 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887029    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-run-netns\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.887336 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887054    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-sysctl-conf\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.887336 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887068    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-daemon-config\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.887336 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887105    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-sysconfig\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.887336 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887132    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-kubernetes\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.887336 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887162    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-sys\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.887336 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887166    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-sysconfig\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.887336 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887190    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-modprobe-d\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.887336 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887199    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-kubernetes\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.887336 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887236    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-sysctl-d\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.887336 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887254    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-sys\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.887336 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887278    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-run-k8s-cni-cncf-io\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.887336 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887305    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-var-lib-cni-multus\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.887336 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887322    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-modprobe-d\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.888103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887338    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a517bba6-536f-4dac-ba8b-2e71964ef8c5-cni-binary-copy\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.888103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887385    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-run-k8s-cni-cncf-io\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.888103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887388    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-sysctl-d\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.888103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887394    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-host\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.888103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887415    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-var-lib-cni-multus\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.888103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887423    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs\") pod \"network-metrics-daemon-7wdcf\" (UID: \"92a21826-d145-4b9b-9897-aaf21f63a67b\") " pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:02.888103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887448    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-systemd\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.888103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887451    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-host\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.888103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/299ac5c9-51d1-4903-b7f1-2001aa7b2bcf-agent-certs\") pod \"konnectivity-agent-r52tw\" (UID: \"299ac5c9-51d1-4903-b7f1-2001aa7b2bcf\") " pod="kube-system/konnectivity-agent-r52tw"
Oct 30 18:36:02.888103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887474    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4a6021d6-6317-473f-97bc-ea023261d90b-serviceca\") pod \"node-ca-qkvdd\" (UID: \"4a6021d6-6317-473f-97bc-ea023261d90b\") " pod="openshift-image-registry/node-ca-qkvdd"
Oct 30 18:36:02.888103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887507    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-cni-dir\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.888103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887516    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-systemd\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.888103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887534    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/299ac5c9-51d1-4903-b7f1-2001aa7b2bcf-konnectivity-ca\") pod \"konnectivity-agent-r52tw\" (UID: \"299ac5c9-51d1-4903-b7f1-2001aa7b2bcf\") " pod="kube-system/konnectivity-agent-r52tw"
Oct 30 18:36:02.888103 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:02.887566    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:02.888103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887593    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-cni-dir\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.888103 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:02.887627    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs podName:92a21826-d145-4b9b-9897-aaf21f63a67b nodeName:}" failed. No retries permitted until 2026-10-30 18:36:03.38760455 +0000 UTC m=+3.168089196 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs") pod "network-metrics-daemon-7wdcf" (UID: "92a21826-d145-4b9b-9897-aaf21f63a67b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:02.888103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887627    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-daemon-config\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.888103 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887665    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-system-cni-dir\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.888781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887720    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-etc-kubernetes\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.888781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887752    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-var-lib-kubelet\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.888781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887783    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-conf-dir\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.888781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887788    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-system-cni-dir\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.888781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-lib-modules\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.888781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887839    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-socket-dir-parent\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.888781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887839    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-conf-dir\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.888781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887840    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-var-lib-kubelet\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.888781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887793    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-etc-kubernetes\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.888781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887904    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnjp7\" (UniqueName: \"kubernetes.io/projected/a517bba6-536f-4dac-ba8b-2e71964ef8c5-kube-api-access-vnjp7\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.888781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887919    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-socket-dir-parent\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.888781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887933    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-var-lib-kubelet\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.888781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887939    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a517bba6-536f-4dac-ba8b-2e71964ef8c5-cni-binary-copy\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.888781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887934    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-lib-modules\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.888781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887963    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2fqp\" (UniqueName: \"kubernetes.io/projected/92a21826-d145-4b9b-9897-aaf21f63a67b-kube-api-access-c2fqp\") pod \"network-metrics-daemon-7wdcf\" (UID: \"92a21826-d145-4b9b-9897-aaf21f63a67b\") " pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:02.888781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887980    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-var-lib-kubelet\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.888781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.887988    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhx9h\" (UniqueName: \"kubernetes.io/projected/d93ec858-ae33-473a-ad4b-18b88664094c-kube-api-access-dhx9h\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.888781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.888008    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npfv7\" (UniqueName: \"kubernetes.io/projected/4a6021d6-6317-473f-97bc-ea023261d90b-kube-api-access-npfv7\") pod \"node-ca-qkvdd\" (UID: \"4a6021d6-6317-473f-97bc-ea023261d90b\") " pod="openshift-image-registry/node-ca-qkvdd"
Oct 30 18:36:02.889291 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.888024    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-hostroot\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.889291 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.888042    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d93ec858-ae33-473a-ad4b-18b88664094c-etc-tuned\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.889291 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.888067    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-var-lib-cni-bin\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.889291 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.888092    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-run\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.889291 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.888165    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/299ac5c9-51d1-4903-b7f1-2001aa7b2bcf-konnectivity-ca\") pod \"konnectivity-agent-r52tw\" (UID: \"299ac5c9-51d1-4903-b7f1-2001aa7b2bcf\") " pod="kube-system/konnectivity-agent-r52tw"
Oct 30 18:36:02.889291 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.888176    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-run\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.889291 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.888213    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-var-lib-cni-bin\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.889291 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.888241    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-hostroot\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.889291 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.889178    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d93ec858-ae33-473a-ad4b-18b88664094c-tmp\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.890281 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.890262    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d93ec858-ae33-473a-ad4b-18b88664094c-etc-tuned\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.890378 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.890362    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/299ac5c9-51d1-4903-b7f1-2001aa7b2bcf-agent-certs\") pod \"konnectivity-agent-r52tw\" (UID: \"299ac5c9-51d1-4903-b7f1-2001aa7b2bcf\") " pod="kube-system/konnectivity-agent-r52tw"
Oct 30 18:36:02.946721 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.946633    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhx9h\" (UniqueName: \"kubernetes.io/projected/d93ec858-ae33-473a-ad4b-18b88664094c-kube-api-access-dhx9h\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:02.946721 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.946685    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npfv7\" (UniqueName: \"kubernetes.io/projected/4a6021d6-6317-473f-97bc-ea023261d90b-kube-api-access-npfv7\") pod \"node-ca-qkvdd\" (UID: \"4a6021d6-6317-473f-97bc-ea023261d90b\") " pod="openshift-image-registry/node-ca-qkvdd"
Oct 30 18:36:02.946721 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.946675    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnjp7\" (UniqueName: \"kubernetes.io/projected/a517bba6-536f-4dac-ba8b-2e71964ef8c5-kube-api-access-vnjp7\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:02.946885 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.946756    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2fqp\" (UniqueName: \"kubernetes.io/projected/92a21826-d145-4b9b-9897-aaf21f63a67b-kube-api-access-c2fqp\") pod \"network-metrics-daemon-7wdcf\" (UID: \"92a21826-d145-4b9b-9897-aaf21f63a67b\") " pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:02.973761 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.973734    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nssbw"
Oct 30 18:36:02.979227 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:02.979207    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nmmwh"
Oct 30 18:36:02.984075 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:02.984049    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7894bc9_1a77_42e3_a59b_926729c93dc9.slice/crio-9e8c259aaafc99b31cb10786e94629f2628ff0a257e257b79dd1858155e43d55 WatchSource:0}: Error finding container 9e8c259aaafc99b31cb10786e94629f2628ff0a257e257b79dd1858155e43d55: Status 404 returned error can't find the container with id 9e8c259aaafc99b31cb10786e94629f2628ff0a257e257b79dd1858155e43d55
Oct 30 18:36:02.987894 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:02.987858    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ebad18_eccb_47c9_b259_67c6d6b84d0f.slice/crio-9156ceacadc3ef5c17ff0a82459481ea6c940b128e65c276ddb10d0106554d53 WatchSource:0}: Error finding container 9156ceacadc3ef5c17ff0a82459481ea6c940b128e65c276ddb10d0106554d53: Status 404 returned error can't find the container with id 9156ceacadc3ef5c17ff0a82459481ea6c940b128e65c276ddb10d0106554d53
Oct 30 18:36:03.010338 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.010316    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-54jpr"
Oct 30 18:36:03.011597 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.011581    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:03.019782 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.019762    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:03.020750 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.020655    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qkvdd"
Oct 30 18:36:03.020980 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:03.020953    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda517bba6_536f_4dac_ba8b_2e71964ef8c5.slice/crio-197507b0c77d583cadd2a8de2b1452d42ccb42e153c5d06b52cde51bb4a195e6 WatchSource:0}: Error finding container 197507b0c77d583cadd2a8de2b1452d42ccb42e153c5d06b52cde51bb4a195e6: Status 404 returned error can't find the container with id 197507b0c77d583cadd2a8de2b1452d42ccb42e153c5d06b52cde51bb4a195e6
Oct 30 18:36:03.022998 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.022974    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:03.023147 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:03.023123    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod114fbcb8_1004_410d_98bc_dc60f8fcae3c.slice/crio-74d238c6a8c7ac178725be85164bcec19e4f1316bab30352419cc161f7577d43 WatchSource:0}: Error finding container 74d238c6a8c7ac178725be85164bcec19e4f1316bab30352419cc161f7577d43: Status 404 returned error can't find the container with id 74d238c6a8c7ac178725be85164bcec19e4f1316bab30352419cc161f7577d43
Oct 30 18:36:03.028438 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.028422    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r52tw"
Oct 30 18:36:03.029736 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:03.029715    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd93ec858_ae33_473a_ad4b_18b88664094c.slice/crio-e72e830a3903689e46746d81b0446f52d2d9fe34b970ee39e0ad5f689b353059 WatchSource:0}: Error finding container e72e830a3903689e46746d81b0446f52d2d9fe34b970ee39e0ad5f689b353059: Status 404 returned error can't find the container with id e72e830a3903689e46746d81b0446f52d2d9fe34b970ee39e0ad5f689b353059
Oct 30 18:36:03.032292 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.032273    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:03.033586 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:03.033559    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a6021d6_6317_473f_97bc_ea023261d90b.slice/crio-36d94e2489c4d23d3321b3cf5a02717ffbd01c4d39715c1e26d42eaf340cdbdc WatchSource:0}: Error finding container 36d94e2489c4d23d3321b3cf5a02717ffbd01c4d39715c1e26d42eaf340cdbdc: Status 404 returned error can't find the container with id 36d94e2489c4d23d3321b3cf5a02717ffbd01c4d39715c1e26d42eaf340cdbdc
Oct 30 18:36:03.037593 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:03.037569    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90c6593d_b40a_43ae_abc0_ce015e48c289.slice/crio-e7dce758fa820d22634e473f8ac42aef98a5118f84adc773ff1426d941333629 WatchSource:0}: Error finding container e7dce758fa820d22634e473f8ac42aef98a5118f84adc773ff1426d941333629: Status 404 returned error can't find the container with id e7dce758fa820d22634e473f8ac42aef98a5118f84adc773ff1426d941333629
Oct 30 18:36:03.042569 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:03.042542    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod299ac5c9_51d1_4903_b7f1_2001aa7b2bcf.slice/crio-de9b062be5610c482edd33d85e74f7bf16a925f7219417dbf1e2c6e9a9b03485 WatchSource:0}: Error finding container de9b062be5610c482edd33d85e74f7bf16a925f7219417dbf1e2c6e9a9b03485: Status 404 returned error can't find the container with id de9b062be5610c482edd33d85e74f7bf16a925f7219417dbf1e2c6e9a9b03485
Oct 30 18:36:03.045529 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:03.045511    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab0116fc_0400_4f7d_be91_27804b24c8f4.slice/crio-3ebae1f01b86ba4f27b38d27f5f5b48f9d53d221f8cf0665322b56c1e4591869 WatchSource:0}: Error finding container 3ebae1f01b86ba4f27b38d27f5f5b48f9d53d221f8cf0665322b56c1e4591869: Status 404 returned error can't find the container with id 3ebae1f01b86ba4f27b38d27f5f5b48f9d53d221f8cf0665322b56c1e4591869
Oct 30 18:36:03.291371 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.291290    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:03.291498 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:03.291435    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:03.291498 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:03.291492    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret podName:f51455e1-dbba-4186-adbf-60f44c9edbe5 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:04.291477324 +0000 UTC m=+4.071961957 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret") pod "global-pull-secret-syncer-b4fj5" (UID: "f51455e1-dbba-4186-adbf-60f44c9edbe5") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:03.392597 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.392558    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs\") pod \"network-metrics-daemon-7wdcf\" (UID: \"92a21826-d145-4b9b-9897-aaf21f63a67b\") " pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:03.392786 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:03.392722    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:03.392786 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.392745    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhn6w\" (UniqueName: \"kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w\") pod \"network-check-target-qk8z2\" (UID: \"f3e9afad-06d1-42eb-b463-5dc5932cd45d\") " pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:03.392786 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:03.392782    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs podName:92a21826-d145-4b9b-9897-aaf21f63a67b nodeName:}" failed. No retries permitted until 2026-10-30 18:36:04.39276635 +0000 UTC m=+4.173250979 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs") pod "network-metrics-daemon-7wdcf" (UID: "92a21826-d145-4b9b-9897-aaf21f63a67b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:03.392915 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:03.392829    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:36:03.392915 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:03.392844    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:36:03.392915 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:03.392853    2581 projected.go:196] Error preparing data for projected volume kube-api-access-fhn6w for pod openshift-network-diagnostics/network-check-target-qk8z2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:03.392915 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:03.392895    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w podName:f3e9afad-06d1-42eb-b463-5dc5932cd45d nodeName:}" failed. No retries permitted until 2026-10-30 18:36:04.392882806 +0000 UTC m=+4.173367435 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fhn6w" (UniqueName: "kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w") pod "network-check-target-qk8z2" (UID: "f3e9afad-06d1-42eb-b463-5dc5932cd45d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:03.769276 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.769165    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:31:03 +0000 UTC" deadline="2028-07-30 06:04:29.524774194 +0000 UTC"
Oct 30 18:36:03.769276 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.769220    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15323h28m25.755558367s"
Oct 30 18:36:03.812819 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.812776    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nmmwh" event={"ID":"d7ebad18-eccb-47c9-b259-67c6d6b84d0f","Type":"ContainerStarted","Data":"9156ceacadc3ef5c17ff0a82459481ea6c940b128e65c276ddb10d0106554d53"}
Oct 30 18:36:03.814382 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.814354    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nssbw" event={"ID":"f7894bc9-1a77-42e3-a59b-926729c93dc9","Type":"ContainerStarted","Data":"9e8c259aaafc99b31cb10786e94629f2628ff0a257e257b79dd1858155e43d55"}
Oct 30 18:36:03.815675 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.815636    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cxhd5" event={"ID":"ab0116fc-0400-4f7d-be91-27804b24c8f4","Type":"ContainerStarted","Data":"3ebae1f01b86ba4f27b38d27f5f5b48f9d53d221f8cf0665322b56c1e4591869"}
Oct 30 18:36:03.817235 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.817202    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r52tw" event={"ID":"299ac5c9-51d1-4903-b7f1-2001aa7b2bcf","Type":"ContainerStarted","Data":"de9b062be5610c482edd33d85e74f7bf16a925f7219417dbf1e2c6e9a9b03485"}
Oct 30 18:36:03.818967 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.818894    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qkvdd" event={"ID":"4a6021d6-6317-473f-97bc-ea023261d90b","Type":"ContainerStarted","Data":"36d94e2489c4d23d3321b3cf5a02717ffbd01c4d39715c1e26d42eaf340cdbdc"}
Oct 30 18:36:03.820924 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.820894    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-78mjc" event={"ID":"d93ec858-ae33-473a-ad4b-18b88664094c","Type":"ContainerStarted","Data":"e72e830a3903689e46746d81b0446f52d2d9fe34b970ee39e0ad5f689b353059"}
Oct 30 18:36:03.822354 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.822327    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz" event={"ID":"90c6593d-b40a-43ae-abc0-ce015e48c289","Type":"ContainerStarted","Data":"e7dce758fa820d22634e473f8ac42aef98a5118f84adc773ff1426d941333629"}
Oct 30 18:36:03.823920 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.823891    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz" event={"ID":"114fbcb8-1004-410d-98bc-dc60f8fcae3c","Type":"ContainerStarted","Data":"74d238c6a8c7ac178725be85164bcec19e4f1316bab30352419cc161f7577d43"}
Oct 30 18:36:03.825558 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:03.825534    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-54jpr" event={"ID":"a517bba6-536f-4dac-ba8b-2e71964ef8c5","Type":"ContainerStarted","Data":"197507b0c77d583cadd2a8de2b1452d42ccb42e153c5d06b52cde51bb4a195e6"}
Oct 30 18:36:04.301468 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:04.301308    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:04.301632 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:04.301533    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:04.301632 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:04.301599    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret podName:f51455e1-dbba-4186-adbf-60f44c9edbe5 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:06.301580575 +0000 UTC m=+6.082065219 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret") pod "global-pull-secret-syncer-b4fj5" (UID: "f51455e1-dbba-4186-adbf-60f44c9edbe5") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:04.402447 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:04.402403    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhn6w\" (UniqueName: \"kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w\") pod \"network-check-target-qk8z2\" (UID: \"f3e9afad-06d1-42eb-b463-5dc5932cd45d\") " pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:04.402623 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:04.402483    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs\") pod \"network-metrics-daemon-7wdcf\" (UID: \"92a21826-d145-4b9b-9897-aaf21f63a67b\") " pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:04.402729 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:04.402662    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:04.402729 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:04.402728    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs podName:92a21826-d145-4b9b-9897-aaf21f63a67b nodeName:}" failed. No retries permitted until 2026-10-30 18:36:06.402708389 +0000 UTC m=+6.183193034 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs") pod "network-metrics-daemon-7wdcf" (UID: "92a21826-d145-4b9b-9897-aaf21f63a67b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:04.402857 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:04.402826    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:36:04.402857 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:04.402841    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:36:04.402857 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:04.402854    2581 projected.go:196] Error preparing data for projected volume kube-api-access-fhn6w for pod openshift-network-diagnostics/network-check-target-qk8z2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:04.403000 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:04.402893    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w podName:f3e9afad-06d1-42eb-b463-5dc5932cd45d nodeName:}" failed. No retries permitted until 2026-10-30 18:36:06.402880996 +0000 UTC m=+6.183365640 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fhn6w" (UniqueName: "kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w") pod "network-check-target-qk8z2" (UID: "f3e9afad-06d1-42eb-b463-5dc5932cd45d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:04.770272 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:04.770178    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:31:03 +0000 UTC" deadline="2028-07-17 22:14:40.144143156 +0000 UTC"
Oct 30 18:36:04.770272 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:04.770226    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15027h38m35.37392172s"
Oct 30 18:36:04.812949 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:04.812913    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:04.813116 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:04.813074    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wdcf" podUID="92a21826-d145-4b9b-9897-aaf21f63a67b"
Oct 30 18:36:04.813177 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:04.813166    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:04.813260 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:04.813240    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4fj5" podUID="f51455e1-dbba-4186-adbf-60f44c9edbe5"
Oct 30 18:36:04.813781 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:04.813763    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:04.813878 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:04.813863    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qk8z2" podUID="f3e9afad-06d1-42eb-b463-5dc5932cd45d"
Oct 30 18:36:06.324421 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:06.324386    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:06.325131 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:06.324645    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:06.325131 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:06.324716    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret podName:f51455e1-dbba-4186-adbf-60f44c9edbe5 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:10.324695685 +0000 UTC m=+10.105180314 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret") pod "global-pull-secret-syncer-b4fj5" (UID: "f51455e1-dbba-4186-adbf-60f44c9edbe5") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:06.425431 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:06.425396    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs\") pod \"network-metrics-daemon-7wdcf\" (UID: \"92a21826-d145-4b9b-9897-aaf21f63a67b\") " pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:06.425604 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:06.425476    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhn6w\" (UniqueName: \"kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w\") pod \"network-check-target-qk8z2\" (UID: \"f3e9afad-06d1-42eb-b463-5dc5932cd45d\") " pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:06.425700 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:06.425616    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:36:06.425700 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:06.425637    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:36:06.425700 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:06.425670    2581 projected.go:196] Error preparing data for projected volume kube-api-access-fhn6w for pod openshift-network-diagnostics/network-check-target-qk8z2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:06.425880 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:06.425731    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w podName:f3e9afad-06d1-42eb-b463-5dc5932cd45d nodeName:}" failed. No retries permitted until 2026-10-30 18:36:10.425712964 +0000 UTC m=+10.206197599 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fhn6w" (UniqueName: "kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w") pod "network-check-target-qk8z2" (UID: "f3e9afad-06d1-42eb-b463-5dc5932cd45d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:06.425880 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:06.425813    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:06.425880 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:06.425847    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs podName:92a21826-d145-4b9b-9897-aaf21f63a67b nodeName:}" failed. No retries permitted until 2026-10-30 18:36:10.425834508 +0000 UTC m=+10.206319153 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs") pod "network-metrics-daemon-7wdcf" (UID: "92a21826-d145-4b9b-9897-aaf21f63a67b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:06.805599 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:06.805566    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:06.805794 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:06.805765    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wdcf" podUID="92a21826-d145-4b9b-9897-aaf21f63a67b"
Oct 30 18:36:06.805874 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:06.805859    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:06.805954 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:06.805937    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4fj5" podUID="f51455e1-dbba-4186-adbf-60f44c9edbe5"
Oct 30 18:36:06.806253 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:06.806126    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:06.806253 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:06.806218    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qk8z2" podUID="f3e9afad-06d1-42eb-b463-5dc5932cd45d"
Oct 30 18:36:06.870758 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:06.870710    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal" event={"ID":"7018e001fd99c1f4ae76683b64500513","Type":"ContainerStarted","Data":"edceb83d17fc64693db7bf63791d26332bcec6c2cc54e58b0610edc21e42f588"}
Oct 30 18:36:07.879038 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:07.879002    2581 generic.go:356] "Generic (PLEG): container finished" podID="7d48fbbc6436568b80ec3382fdeed930" containerID="05994d6179cee66ea6ec258129e44cb3cf95e525fa4a2f6da3778f6c700730ef" exitCode=0
Oct 30 18:36:07.879427 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:07.879097    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal" event={"ID":"7d48fbbc6436568b80ec3382fdeed930","Type":"ContainerDied","Data":"05994d6179cee66ea6ec258129e44cb3cf95e525fa4a2f6da3778f6c700730ef"}
Oct 30 18:36:07.883907 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:07.882905    2581 generic.go:356] "Generic (PLEG): container finished" podID="ab0116fc-0400-4f7d-be91-27804b24c8f4" containerID="dd4ce462f2d806b8c19db8ca9309f64882797f045063c171cfbd650da3a66ba0" exitCode=0
Oct 30 18:36:07.883907 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:07.883782    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cxhd5" event={"ID":"ab0116fc-0400-4f7d-be91-27804b24c8f4","Type":"ContainerDied","Data":"dd4ce462f2d806b8c19db8ca9309f64882797f045063c171cfbd650da3a66ba0"}
Oct 30 18:36:08.097480 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:08.097414    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal" podStartSLOduration=6.0973949 podStartE2EDuration="6.0973949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:08.096920303 +0000 UTC m=+7.877404953" watchObservedRunningTime="2026-10-30 18:36:08.0973949 +0000 UTC m=+7.877879551"
Oct 30 18:36:08.806850 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:08.806776    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:08.806850 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:08.806789    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:08.807066 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:08.806957    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qk8z2" podUID="f3e9afad-06d1-42eb-b463-5dc5932cd45d"
Oct 30 18:36:08.807389 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:08.807155    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:08.807389 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:08.807300    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wdcf" podUID="92a21826-d145-4b9b-9897-aaf21f63a67b"
Oct 30 18:36:08.807389 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:08.807365    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4fj5" podUID="f51455e1-dbba-4186-adbf-60f44c9edbe5"
Oct 30 18:36:08.890741 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:08.890081    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal" event={"ID":"7d48fbbc6436568b80ec3382fdeed930","Type":"ContainerStarted","Data":"71bef265f12a68f9ea81389338f12da88bc07185618bfbdd72c4dda0be9eeea7"}
Oct 30 18:36:08.898058 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:08.897984    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r52tw" event={"ID":"299ac5c9-51d1-4903-b7f1-2001aa7b2bcf","Type":"ContainerStarted","Data":"47ab46dd24fcd36545d48c833f4f0ff6241cdfd83996904c63a50e0fd708998e"}
Oct 30 18:36:09.105467 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:09.104233    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal" podStartSLOduration=7.104213879 podStartE2EDuration="7.104213879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:08.997534132 +0000 UTC m=+8.778018781" watchObservedRunningTime="2026-10-30 18:36:09.104213879 +0000 UTC m=+8.884698528"
Oct 30 18:36:09.901842 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:09.901773    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz" event={"ID":"90c6593d-b40a-43ae-abc0-ce015e48c289","Type":"ContainerStarted","Data":"4f2d30976c0b773437caeeabb116272d256249d39ca9ee9c7721015e6250898e"}
Oct 30 18:36:09.905571 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:09.905543    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qkvdd" event={"ID":"4a6021d6-6317-473f-97bc-ea023261d90b","Type":"ContainerStarted","Data":"81140da536da05e0c5a2e6be60688409dab5f28a6c9380fce4a86aed15694dee"}
Oct 30 18:36:10.011053 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:10.010994    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-r52tw" podStartSLOduration=4.057352308 podStartE2EDuration="9.010973351s" totalImagesPullingTime="4.953621043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:09.105446771 +0000 UTC m=+8.885931433" watchObservedRunningTime="2026-10-30 18:36:10.010973351 +0000 UTC m=+9.791458001"
Oct 30 18:36:10.363976 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:10.363940    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:10.364155 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:10.364108    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:10.364216 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:10.364167    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret podName:f51455e1-dbba-4186-adbf-60f44c9edbe5 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:18.36414892 +0000 UTC m=+18.144633547 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret") pod "global-pull-secret-syncer-b4fj5" (UID: "f51455e1-dbba-4186-adbf-60f44c9edbe5") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:10.465261 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:10.465224    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhn6w\" (UniqueName: \"kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w\") pod \"network-check-target-qk8z2\" (UID: \"f3e9afad-06d1-42eb-b463-5dc5932cd45d\") " pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:10.465426 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:10.465288    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs\") pod \"network-metrics-daemon-7wdcf\" (UID: \"92a21826-d145-4b9b-9897-aaf21f63a67b\") " pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:10.465501 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:10.465423    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:10.465501 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:10.465478    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs podName:92a21826-d145-4b9b-9897-aaf21f63a67b nodeName:}" failed. No retries permitted until 2026-10-30 18:36:18.465460859 +0000 UTC m=+18.245945519 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs") pod "network-metrics-daemon-7wdcf" (UID: "92a21826-d145-4b9b-9897-aaf21f63a67b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:10.465634 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:10.465562    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:36:10.465634 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:10.465577    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:36:10.465634 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:10.465588    2581 projected.go:196] Error preparing data for projected volume kube-api-access-fhn6w for pod openshift-network-diagnostics/network-check-target-qk8z2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:10.465773 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:10.465640    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w podName:f3e9afad-06d1-42eb-b463-5dc5932cd45d nodeName:}" failed. No retries permitted until 2026-10-30 18:36:18.465628188 +0000 UTC m=+18.246112815 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fhn6w" (UniqueName: "kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w") pod "network-check-target-qk8z2" (UID: "f3e9afad-06d1-42eb-b463-5dc5932cd45d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:10.808070 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:10.808039    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:10.808247 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:10.808216    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wdcf" podUID="92a21826-d145-4b9b-9897-aaf21f63a67b"
Oct 30 18:36:10.808358 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:10.808344    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:10.808417 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:10.808401    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4fj5" podUID="f51455e1-dbba-4186-adbf-60f44c9edbe5"
Oct 30 18:36:10.808470 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:10.808434    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:10.808660 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:10.808643    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qk8z2" podUID="f3e9afad-06d1-42eb-b463-5dc5932cd45d"
Oct 30 18:36:11.893473 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:11.893420    2581 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 18:36:11.913172 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:11.912729    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-54jpr" event={"ID":"a517bba6-536f-4dac-ba8b-2e71964ef8c5","Type":"ContainerStarted","Data":"f24d2104223e547a41227d29dcf727cb5082fb135ceec5e73cabc98310466e28"}
Oct 30 18:36:11.916569 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:11.916540    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz" event={"ID":"90c6593d-b40a-43ae-abc0-ce015e48c289","Type":"ContainerStarted","Data":"f87d6cbb54ed654295b6faaa3898ea0375b43550a8848b9b2ae935fbcc044f21"}
Oct 30 18:36:12.030116 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:12.030059    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qkvdd" podStartSLOduration=4.979093409 podStartE2EDuration="11.030041551s" totalImagesPullingTime="6.050948142s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:10.011976294 +0000 UTC m=+9.792460944" watchObservedRunningTime="2026-10-30 18:36:12.030041551 +0000 UTC m=+11.810526200"
Oct 30 18:36:12.030780 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:12.030750    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-54jpr" podStartSLOduration=2.322375715 podStartE2EDuration="11.030740083s" totalImagesPullingTime="8.708364368s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:12.029988801 +0000 UTC m=+11.810473451" watchObservedRunningTime="2026-10-30 18:36:12.030740083 +0000 UTC m=+11.811224732"
Oct 30 18:36:12.037440 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:12.037398    2581 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-r52tw"
Oct 30 18:36:12.038845 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:12.038826    2581 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-r52tw"
Oct 30 18:36:12.783752 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:12.783395    2581 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:36:11.893442267Z","UUID":"c1b9144b-e2a0-4553-b02c-5028b832106b","Handler":null,"Name":"","Endpoint":""}
Oct 30 18:36:12.785016 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:12.784994    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 18:36:12.785128 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:12.785023    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 18:36:12.805386 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:12.805254    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:12.805386 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:12.805271    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:12.805386 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:12.805353    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4fj5" podUID="f51455e1-dbba-4186-adbf-60f44c9edbe5"
Oct 30 18:36:12.805626 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:12.805436    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wdcf" podUID="92a21826-d145-4b9b-9897-aaf21f63a67b"
Oct 30 18:36:12.805626 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:12.805454    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:12.805626 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:12.805528    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qk8z2" podUID="f3e9afad-06d1-42eb-b463-5dc5932cd45d"
Oct 30 18:36:12.922472 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:12.921737    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nmmwh" event={"ID":"d7ebad18-eccb-47c9-b259-67c6d6b84d0f","Type":"ContainerStarted","Data":"93b13b4eb07b236a6942423d6a293bb5e90178d59505dab3c3a26920dc5439f9"}
Oct 30 18:36:12.924642 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:12.924485    2581 generic.go:356] "Generic (PLEG): container finished" podID="ab0116fc-0400-4f7d-be91-27804b24c8f4" containerID="b21965d88dfebf7911fb927d0aed491c9a10f2bdf8478fee636617e96fae8c93" exitCode=0
Oct 30 18:36:12.924642 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:12.924563    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cxhd5" event={"ID":"ab0116fc-0400-4f7d-be91-27804b24c8f4","Type":"ContainerDied","Data":"b21965d88dfebf7911fb927d0aed491c9a10f2bdf8478fee636617e96fae8c93"}
Oct 30 18:36:13.140710 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:13.140656    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-nmmwh" podStartSLOduration=3.140415688 podStartE2EDuration="12.140641705s" totalImagesPullingTime="9.000226017s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:13.028673243 +0000 UTC m=+12.809157892" watchObservedRunningTime="2026-10-30 18:36:13.140641705 +0000 UTC m=+12.921126354"
Oct 30 18:36:13.644387 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:13.644361    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-r52tw"
Oct 30 18:36:13.645282 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:13.645259    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-r52tw"
Oct 30 18:36:13.929314 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:13.929278    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz" event={"ID":"90c6593d-b40a-43ae-abc0-ce015e48c289","Type":"ContainerStarted","Data":"0d0aebc044a387ec2e76afd0cb3efb0975007bc27c3800260d2cd323560191a2"}
Oct 30 18:36:13.931026 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:13.930998    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nssbw" event={"ID":"f7894bc9-1a77-42e3-a59b-926729c93dc9","Type":"ContainerStarted","Data":"7030e46d2bc992f6ca587a05daf972e7d1a6ef5a5eff0eb0eb536c83cf2f10d5"}
Oct 30 18:36:14.038355 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:14.038298    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz" podStartSLOduration=2.678218819 podStartE2EDuration="13.038283751s" totalImagesPullingTime="10.360064932s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:14.038011722 +0000 UTC m=+13.818496373" watchObservedRunningTime="2026-10-30 18:36:14.038283751 +0000 UTC m=+13.818768399"
Oct 30 18:36:14.148451 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:14.148351    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nssbw" podStartSLOduration=4.052167191 podStartE2EDuration="13.148333081s" totalImagesPullingTime="9.09616589s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:14.148216477 +0000 UTC m=+13.928701127" watchObservedRunningTime="2026-10-30 18:36:14.148333081 +0000 UTC m=+13.928817732"
Oct 30 18:36:14.805456 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:14.805374    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:14.805456 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:14.805391    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:14.805685 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:14.805536    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4fj5" podUID="f51455e1-dbba-4186-adbf-60f44c9edbe5"
Oct 30 18:36:14.805685 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:14.805582    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:14.805811 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:14.805699    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qk8z2" podUID="f3e9afad-06d1-42eb-b463-5dc5932cd45d"
Oct 30 18:36:14.805811 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:14.805782    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wdcf" podUID="92a21826-d145-4b9b-9897-aaf21f63a67b"
Oct 30 18:36:14.935161 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:14.935123    2581 generic.go:356] "Generic (PLEG): container finished" podID="ab0116fc-0400-4f7d-be91-27804b24c8f4" containerID="f43c14c280c7f015f06d54695f2ee9572fa9b0d833e56267b009808ffad8d6d8" exitCode=0
Oct 30 18:36:14.935806 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:14.935203    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cxhd5" event={"ID":"ab0116fc-0400-4f7d-be91-27804b24c8f4","Type":"ContainerDied","Data":"f43c14c280c7f015f06d54695f2ee9572fa9b0d833e56267b009808ffad8d6d8"}
Oct 30 18:36:14.937700 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:14.937657    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-78mjc" event={"ID":"d93ec858-ae33-473a-ad4b-18b88664094c","Type":"ContainerStarted","Data":"264b7f562ab7638471d6d7ae568c7631a6085a6bde5b3c22dd56cca828344ffe"}
Oct 30 18:36:16.806055 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:16.806022    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:16.806055 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:16.806047    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:16.806596 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:16.806128    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qk8z2" podUID="f3e9afad-06d1-42eb-b463-5dc5932cd45d"
Oct 30 18:36:16.806596 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:16.806203    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:16.806596 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:16.806316    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wdcf" podUID="92a21826-d145-4b9b-9897-aaf21f63a67b"
Oct 30 18:36:16.806596 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:16.806396    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4fj5" podUID="f51455e1-dbba-4186-adbf-60f44c9edbe5"
Oct 30 18:36:16.944710 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:16.944673    2581 generic.go:356] "Generic (PLEG): container finished" podID="ab0116fc-0400-4f7d-be91-27804b24c8f4" containerID="5d632c16162f5c46b761a49a0c3d7a0ba06b6083fd5f5cbdaa25fbd39f32e93d" exitCode=0
Oct 30 18:36:16.944846 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:16.944720    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cxhd5" event={"ID":"ab0116fc-0400-4f7d-be91-27804b24c8f4","Type":"ContainerDied","Data":"5d632c16162f5c46b761a49a0c3d7a0ba06b6083fd5f5cbdaa25fbd39f32e93d"}
Oct 30 18:36:17.146334 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:17.146221    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-78mjc" podStartSLOduration=4.646762884 podStartE2EDuration="16.146206644s" totalImagesPullingTime="11.49944376s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:15.15614839 +0000 UTC m=+14.936633039" watchObservedRunningTime="2026-10-30 18:36:17.146206644 +0000 UTC m=+16.926691293"
Oct 30 18:36:18.424416 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:18.424382    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:18.424762 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:18.424529    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:18.424762 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:18.424587    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret podName:f51455e1-dbba-4186-adbf-60f44c9edbe5 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:34.424571952 +0000 UTC m=+34.205056596 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret") pod "global-pull-secret-syncer-b4fj5" (UID: "f51455e1-dbba-4186-adbf-60f44c9edbe5") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:18.524810 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:18.524741    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs\") pod \"network-metrics-daemon-7wdcf\" (UID: \"92a21826-d145-4b9b-9897-aaf21f63a67b\") " pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:18.524892 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:18.524834    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhn6w\" (UniqueName: \"kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w\") pod \"network-check-target-qk8z2\" (UID: \"f3e9afad-06d1-42eb-b463-5dc5932cd45d\") " pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:18.524892 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:18.524860    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:18.524960 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:18.524927    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs podName:92a21826-d145-4b9b-9897-aaf21f63a67b nodeName:}" failed. No retries permitted until 2026-10-30 18:36:34.524908063 +0000 UTC m=+34.305392719 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs") pod "network-metrics-daemon-7wdcf" (UID: "92a21826-d145-4b9b-9897-aaf21f63a67b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:18.525016 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:18.524975    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:36:18.525016 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:18.524997    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:36:18.525016 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:18.525010    2581 projected.go:196] Error preparing data for projected volume kube-api-access-fhn6w for pod openshift-network-diagnostics/network-check-target-qk8z2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:18.525113 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:18.525065    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w podName:f3e9afad-06d1-42eb-b463-5dc5932cd45d nodeName:}" failed. No retries permitted until 2026-10-30 18:36:34.52505028 +0000 UTC m=+34.305534924 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fhn6w" (UniqueName: "kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w") pod "network-check-target-qk8z2" (UID: "f3e9afad-06d1-42eb-b463-5dc5932cd45d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:18.805308 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:18.805276    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:18.805496 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:18.805318    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:18.805496 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:18.805385    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qk8z2" podUID="f3e9afad-06d1-42eb-b463-5dc5932cd45d"
Oct 30 18:36:18.805496 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:18.805282    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:18.805496 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:18.805487    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wdcf" podUID="92a21826-d145-4b9b-9897-aaf21f63a67b"
Oct 30 18:36:18.805718 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:18.805558    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4fj5" podUID="f51455e1-dbba-4186-adbf-60f44c9edbe5"
Oct 30 18:36:18.952064 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:18.952045    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 18:36:18.952412 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:18.952392    2581 generic.go:356] "Generic (PLEG): container finished" podID="114fbcb8-1004-410d-98bc-dc60f8fcae3c" containerID="03b05f32b6c8e5315b71345369389005b85e9ec8d87fbec3189b0cb8fdc488e3" exitCode=1
Oct 30 18:36:18.952517 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:18.952433    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz" event={"ID":"114fbcb8-1004-410d-98bc-dc60f8fcae3c","Type":"ContainerStarted","Data":"adf418cae0935464c2732bd5aced8994bba6b9faad941b3bc77eb875006b9bea"}
Oct 30 18:36:18.952517 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:18.952510    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz" event={"ID":"114fbcb8-1004-410d-98bc-dc60f8fcae3c","Type":"ContainerStarted","Data":"cb897cc6b171375625097aa7b249c649d9dc370bd9b364801f4dac69d5872f9e"}
Oct 30 18:36:18.952638 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:18.952561    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz" event={"ID":"114fbcb8-1004-410d-98bc-dc60f8fcae3c","Type":"ContainerStarted","Data":"f34899181cd7cd40e9a9bacffb624d6b5de4a2c7464ff34a3aa1dcdda96d08d0"}
Oct 30 18:36:18.952638 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:18.952625    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz" event={"ID":"114fbcb8-1004-410d-98bc-dc60f8fcae3c","Type":"ContainerStarted","Data":"32cf69d1bd6cf5253cab0a656966b765319231b5e64a0a9402f63f4c1bd61c90"}
Oct 30 18:36:18.952740 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:18.952675    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz" event={"ID":"114fbcb8-1004-410d-98bc-dc60f8fcae3c","Type":"ContainerDied","Data":"03b05f32b6c8e5315b71345369389005b85e9ec8d87fbec3189b0cb8fdc488e3"}
Oct 30 18:36:18.952740 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:18.952724    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz" event={"ID":"114fbcb8-1004-410d-98bc-dc60f8fcae3c","Type":"ContainerStarted","Data":"ad5defaac32e199e535ea49eb0cf218385b807625007c07c607ae08fc1cfb904"}
Oct 30 18:36:19.956765 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:19.956736    2581 generic.go:356] "Generic (PLEG): container finished" podID="ab0116fc-0400-4f7d-be91-27804b24c8f4" containerID="34b3448b05c4a5dae8f7d6a1e438d2a3fba26544d449c8dd757f6df3b679f907" exitCode=0
Oct 30 18:36:19.957212 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:19.956778    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cxhd5" event={"ID":"ab0116fc-0400-4f7d-be91-27804b24c8f4","Type":"ContainerDied","Data":"34b3448b05c4a5dae8f7d6a1e438d2a3fba26544d449c8dd757f6df3b679f907"}
Oct 30 18:36:20.807956 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:20.807928    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:20.808078 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:20.808005    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:20.808078 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:20.808032    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:20.808154 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:20.808126    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wdcf" podUID="92a21826-d145-4b9b-9897-aaf21f63a67b"
Oct 30 18:36:20.808189 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:20.808160    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qk8z2" podUID="f3e9afad-06d1-42eb-b463-5dc5932cd45d"
Oct 30 18:36:20.808244 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:20.808224    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4fj5" podUID="f51455e1-dbba-4186-adbf-60f44c9edbe5"
Oct 30 18:36:20.961337 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:20.961276    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 18:36:20.961844 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:20.961633    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz" event={"ID":"114fbcb8-1004-410d-98bc-dc60f8fcae3c","Type":"ContainerStarted","Data":"5bbb749663319067d9d832d091f5002247084c90e6168180489e6b68ce79ce53"}
Oct 30 18:36:20.964366 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:20.964341    2581 generic.go:356] "Generic (PLEG): container finished" podID="ab0116fc-0400-4f7d-be91-27804b24c8f4" containerID="91a13660483839a894e8962d0616b503a04e946757cc61ca39ac4eb81a703f81" exitCode=0
Oct 30 18:36:20.964481 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:20.964381    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cxhd5" event={"ID":"ab0116fc-0400-4f7d-be91-27804b24c8f4","Type":"ContainerDied","Data":"91a13660483839a894e8962d0616b503a04e946757cc61ca39ac4eb81a703f81"}
Oct 30 18:36:21.969102 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:21.969070    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cxhd5" event={"ID":"ab0116fc-0400-4f7d-be91-27804b24c8f4","Type":"ContainerStarted","Data":"3eeb9111eb8d6363331e6349ee80227702829bdeac91a17f8e1b4d50ddfcad69"}
Oct 30 18:36:22.170650 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:22.170585    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cxhd5" podStartSLOduration=8.214560539 podStartE2EDuration="21.170571276s" totalImagesPullingTime="12.956010737s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:01 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:22.170570791 +0000 UTC m=+21.951055436" watchObservedRunningTime="2026-10-30 18:36:22.170571276 +0000 UTC m=+21.951055924"
Oct 30 18:36:22.808366 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:22.808229    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:22.808513 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:22.808271    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:22.808513 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:22.808472    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wdcf" podUID="92a21826-d145-4b9b-9897-aaf21f63a67b"
Oct 30 18:36:22.808513 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:22.808284    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:22.808669 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:22.808530    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4fj5" podUID="f51455e1-dbba-4186-adbf-60f44c9edbe5"
Oct 30 18:36:22.808669 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:22.808562    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qk8z2" podUID="f3e9afad-06d1-42eb-b463-5dc5932cd45d"
Oct 30 18:36:22.974244 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:22.974149    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 18:36:22.974783 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:22.974502    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz" event={"ID":"114fbcb8-1004-410d-98bc-dc60f8fcae3c","Type":"ContainerStarted","Data":"2745b360ebea9ed1294f3b348477b3221eef35a947994189d48300cbdf9e213f"}
Oct 30 18:36:22.975094 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:22.975074    2581 scope.go:122] "RemoveContainer" containerID="03b05f32b6c8e5315b71345369389005b85e9ec8d87fbec3189b0cb8fdc488e3"
Oct 30 18:36:23.980482 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:23.980460    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 18:36:23.980945 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:23.980820    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz" event={"ID":"114fbcb8-1004-410d-98bc-dc60f8fcae3c","Type":"ContainerStarted","Data":"cd50a2f74275fbab7bc5dd451d91c0069705e3b4575212f41b372c8dfd0e4c42"}
Oct 30 18:36:23.980945 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:23.980924    2581 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:36:23.981326 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:23.981298    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:23.981414 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:23.981373    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:23.995030 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:23.995000    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:23.995337 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:23.995323    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:23.999449 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:23.999431    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:24.234829 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:24.234743    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz" podStartSLOduration=8.080514645 podStartE2EDuration="23.234729463s" totalImagesPullingTime="15.154214818s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:24.188505067 +0000 UTC m=+23.968989715" watchObservedRunningTime="2026-10-30 18:36:24.234729463 +0000 UTC m=+24.015214112"
Oct 30 18:36:24.805639 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:24.805594    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:24.805776 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:24.805646    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:24.805776 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:24.805613    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:24.805776 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:24.805738    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wdcf" podUID="92a21826-d145-4b9b-9897-aaf21f63a67b"
Oct 30 18:36:24.805897 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:24.805878    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4fj5" podUID="f51455e1-dbba-4186-adbf-60f44c9edbe5"
Oct 30 18:36:24.806004 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:24.805976    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qk8z2" podUID="f3e9afad-06d1-42eb-b463-5dc5932cd45d"
Oct 30 18:36:25.261983 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:25.261903    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b4fj5"]
Oct 30 18:36:25.262420 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:25.262020    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:25.262420 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:25.262117    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4fj5" podUID="f51455e1-dbba-4186-adbf-60f44c9edbe5"
Oct 30 18:36:25.264692 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:25.264670    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qk8z2"]
Oct 30 18:36:25.264839 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:25.264824    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:25.264970 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:25.264953    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qk8z2" podUID="f3e9afad-06d1-42eb-b463-5dc5932cd45d"
Oct 30 18:36:25.265332 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:25.265312    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7wdcf"]
Oct 30 18:36:25.265427 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:25.265406    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:25.265512 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:25.265495    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wdcf" podUID="92a21826-d145-4b9b-9897-aaf21f63a67b"
Oct 30 18:36:26.805502 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:26.805477    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:26.805842 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:26.805484    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:26.805842 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:26.805583    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4fj5" podUID="f51455e1-dbba-4186-adbf-60f44c9edbe5"
Oct 30 18:36:26.805842 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:26.805485    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:26.805842 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:26.805692    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qk8z2" podUID="f3e9afad-06d1-42eb-b463-5dc5932cd45d"
Oct 30 18:36:26.805842 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:26.805790    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wdcf" podUID="92a21826-d145-4b9b-9897-aaf21f63a67b"
Oct 30 18:36:28.805363 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:28.805329    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:28.805363 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:28.805365    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:28.805910 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:28.805329    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:28.805910 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:28.805437    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4fj5" podUID="f51455e1-dbba-4186-adbf-60f44c9edbe5"
Oct 30 18:36:28.805910 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:28.805524    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qk8z2" podUID="f3e9afad-06d1-42eb-b463-5dc5932cd45d"
Oct 30 18:36:28.805910 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:28.805636    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wdcf" podUID="92a21826-d145-4b9b-9897-aaf21f63a67b"
Oct 30 18:36:30.460409 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.460163    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeReady"
Oct 30 18:36:30.460719 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.460506    2581 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 18:36:30.564769 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.564733    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hwxx2"]
Oct 30 18:36:30.566598 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.566582    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zmj6x"]
Oct 30 18:36:30.566725 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.566710    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hwxx2"
Oct 30 18:36:30.568338 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.568320    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:30.607028 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.607004    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hwxx2"]
Oct 30 18:36:30.615417 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.615395    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 18:36:30.615633 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.615614    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 18:36:30.615709 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.615661    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 18:36:30.615709 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.615665    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-zmfwz\""
Oct 30 18:36:30.616235 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.616221    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 18:36:30.616280 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.616246    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 18:36:30.616337 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.616327    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vtx29\""
Oct 30 18:36:30.667555 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.667520    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zmj6x"]
Oct 30 18:36:30.717073 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.717015    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5db1f514-bc37-424a-89ce-71dbc6a520e9-config-volume\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:30.717073 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.717053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f98872b6-9c55-467c-9c70-2831de3fb841-cert\") pod \"ingress-canary-hwxx2\" (UID: \"f98872b6-9c55-467c-9c70-2831de3fb841\") " pod="openshift-ingress-canary/ingress-canary-hwxx2"
Oct 30 18:36:30.717252 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.717088    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn988\" (UniqueName: \"kubernetes.io/projected/f98872b6-9c55-467c-9c70-2831de3fb841-kube-api-access-vn988\") pod \"ingress-canary-hwxx2\" (UID: \"f98872b6-9c55-467c-9c70-2831de3fb841\") " pod="openshift-ingress-canary/ingress-canary-hwxx2"
Oct 30 18:36:30.717252 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.717216    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j82vx\" (UniqueName: \"kubernetes.io/projected/5db1f514-bc37-424a-89ce-71dbc6a520e9-kube-api-access-j82vx\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:30.717354 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.717261    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:30.717354 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.717291    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5db1f514-bc37-424a-89ce-71dbc6a520e9-tmp-dir\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:30.806510 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.806482    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:30.806670 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.806655    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:30.806800 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.806783    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:30.817643 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.817629    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:30.817766 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.817653    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5db1f514-bc37-424a-89ce-71dbc6a520e9-tmp-dir\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:30.817766 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.817682    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5db1f514-bc37-424a-89ce-71dbc6a520e9-config-volume\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:30.817766 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.817710    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f98872b6-9c55-467c-9c70-2831de3fb841-cert\") pod \"ingress-canary-hwxx2\" (UID: \"f98872b6-9c55-467c-9c70-2831de3fb841\") " pod="openshift-ingress-canary/ingress-canary-hwxx2"
Oct 30 18:36:30.817766 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.817742    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn988\" (UniqueName: \"kubernetes.io/projected/f98872b6-9c55-467c-9c70-2831de3fb841-kube-api-access-vn988\") pod \"ingress-canary-hwxx2\" (UID: \"f98872b6-9c55-467c-9c70-2831de3fb841\") " pod="openshift-ingress-canary/ingress-canary-hwxx2"
Oct 30 18:36:30.817987 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:30.817771    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:36:30.817987 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.817807    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j82vx\" (UniqueName: \"kubernetes.io/projected/5db1f514-bc37-424a-89ce-71dbc6a520e9-kube-api-access-j82vx\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:30.817987 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:30.817826    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls podName:5db1f514-bc37-424a-89ce-71dbc6a520e9 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:31.31781264 +0000 UTC m=+31.098297268 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls") pod "dns-default-zmj6x" (UID: "5db1f514-bc37-424a-89ce-71dbc6a520e9") : secret "dns-default-metrics-tls" not found
Oct 30 18:36:30.818234 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.818199    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5db1f514-bc37-424a-89ce-71dbc6a520e9-tmp-dir\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:30.818615 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.818578    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5db1f514-bc37-424a-89ce-71dbc6a520e9-config-volume\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:30.822193 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.822175    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f98872b6-9c55-467c-9c70-2831de3fb841-cert\") pod \"ingress-canary-hwxx2\" (UID: \"f98872b6-9c55-467c-9c70-2831de3fb841\") " pod="openshift-ingress-canary/ingress-canary-hwxx2"
Oct 30 18:36:30.856570 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.856547    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 18:36:30.856676 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.856643    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 18:36:30.856676 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.856669    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 18:36:30.857091 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.857066    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-5slhp\""
Oct 30 18:36:30.857153 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.857098    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cxkbs\""
Oct 30 18:36:30.857153 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.857132    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-9ph7g\""
Oct 30 18:36:30.857229 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.857188    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 18:36:30.871896 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.871874    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j82vx\" (UniqueName: \"kubernetes.io/projected/5db1f514-bc37-424a-89ce-71dbc6a520e9-kube-api-access-j82vx\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:30.873108 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.873093    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn988\" (UniqueName: \"kubernetes.io/projected/f98872b6-9c55-467c-9c70-2831de3fb841-kube-api-access-vn988\") pod \"ingress-canary-hwxx2\" (UID: \"f98872b6-9c55-467c-9c70-2831de3fb841\") " pod="openshift-ingress-canary/ingress-canary-hwxx2"
Oct 30 18:36:30.879871 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:30.879855    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hwxx2"
Oct 30 18:36:31.248521 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:31.248493    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hwxx2"]
Oct 30 18:36:31.252724 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:31.252699    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf98872b6_9c55_467c_9c70_2831de3fb841.slice/crio-7cbbb3a060e45b04b442cd03dd28b9df91f2132707a0bcf475d14a91a6ae994b WatchSource:0}: Error finding container 7cbbb3a060e45b04b442cd03dd28b9df91f2132707a0bcf475d14a91a6ae994b: Status 404 returned error can't find the container with id 7cbbb3a060e45b04b442cd03dd28b9df91f2132707a0bcf475d14a91a6ae994b
Oct 30 18:36:31.322973 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:31.322949    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:31.323118 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:31.323100    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:36:31.323184 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:31.323166    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls podName:5db1f514-bc37-424a-89ce-71dbc6a520e9 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:32.323147404 +0000 UTC m=+32.103632032 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls") pod "dns-default-zmj6x" (UID: "5db1f514-bc37-424a-89ce-71dbc6a520e9") : secret "dns-default-metrics-tls" not found
Oct 30 18:36:31.999141 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:31.999112    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hwxx2" event={"ID":"f98872b6-9c55-467c-9c70-2831de3fb841","Type":"ContainerStarted","Data":"7cbbb3a060e45b04b442cd03dd28b9df91f2132707a0bcf475d14a91a6ae994b"}
Oct 30 18:36:32.330038 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:32.330002    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:32.330192 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:32.330156    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:36:32.330235 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:32.330217    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls podName:5db1f514-bc37-424a-89ce-71dbc6a520e9 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:34.330203076 +0000 UTC m=+34.110687726 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls") pod "dns-default-zmj6x" (UID: "5db1f514-bc37-424a-89ce-71dbc6a520e9") : secret "dns-default-metrics-tls" not found
Oct 30 18:36:34.263299 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.263266    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-nr9vn"]
Oct 30 18:36:34.267693 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.267671    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nr9vn"
Oct 30 18:36:34.316531 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.316506    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:34.343490 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.343458    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz7ff\" (UniqueName: \"kubernetes.io/projected/9afdc202-0860-42d2-99dc-e1e028b75953-kube-api-access-sz7ff\") pod \"migrator-66df5d496b-nr9vn\" (UID: \"9afdc202-0860-42d2-99dc-e1e028b75953\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nr9vn"
Oct 30 18:36:34.343633 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.343504    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:34.343690 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:34.343643    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:36:34.343732 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:34.343692    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls podName:5db1f514-bc37-424a-89ce-71dbc6a520e9 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:38.343675533 +0000 UTC m=+38.124160160 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls") pod "dns-default-zmj6x" (UID: "5db1f514-bc37-424a-89ce-71dbc6a520e9") : secret "dns-default-metrics-tls" not found
Oct 30 18:36:34.362304 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.362270    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-mbxkm\""
Oct 30 18:36:34.362304 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.362283    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 18:36:34.363485 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.363459    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-nr9vn"]
Oct 30 18:36:34.444859 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.444819    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:34.445013 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.444884    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz7ff\" (UniqueName: \"kubernetes.io/projected/9afdc202-0860-42d2-99dc-e1e028b75953-kube-api-access-sz7ff\") pod \"migrator-66df5d496b-nr9vn\" (UID: \"9afdc202-0860-42d2-99dc-e1e028b75953\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nr9vn"
Oct 30 18:36:34.447362 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.447331    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:34.499067 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.499035    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz7ff\" (UniqueName: \"kubernetes.io/projected/9afdc202-0860-42d2-99dc-e1e028b75953-kube-api-access-sz7ff\") pod \"migrator-66df5d496b-nr9vn\" (UID: \"9afdc202-0860-42d2-99dc-e1e028b75953\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nr9vn"
Oct 30 18:36:34.546075 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.546044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhn6w\" (UniqueName: \"kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w\") pod \"network-check-target-qk8z2\" (UID: \"f3e9afad-06d1-42eb-b463-5dc5932cd45d\") " pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:34.546236 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.546089    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs\") pod \"network-metrics-daemon-7wdcf\" (UID: \"92a21826-d145-4b9b-9897-aaf21f63a67b\") " pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:34.546236 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:34.546206    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 18:36:34.546347 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:34.546264    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs podName:92a21826-d145-4b9b-9897-aaf21f63a67b nodeName:}" failed. No retries permitted until 2026-10-30 18:37:06.546247901 +0000 UTC m=+66.326732533 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs") pod "network-metrics-daemon-7wdcf" (UID: "92a21826-d145-4b9b-9897-aaf21f63a67b") : secret "metrics-daemon-secret" not found
Oct 30 18:36:34.548519 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.548495    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhn6w\" (UniqueName: \"kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w\") pod \"network-check-target-qk8z2\" (UID: \"f3e9afad-06d1-42eb-b463-5dc5932cd45d\") " pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:34.578454 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.578424    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nr9vn"
Oct 30 18:36:34.717173 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.717141    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:34.729253 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.729235    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:34.939188 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:34.939154    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-nr9vn"]
Oct 30 18:36:34.942795 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:34.942773    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9afdc202_0860_42d2_99dc_e1e028b75953.slice/crio-a872df7cb2015d7d1c0dca0187e93c6cd6c58d20446ba442a6d9daf70371177e WatchSource:0}: Error finding container a872df7cb2015d7d1c0dca0187e93c6cd6c58d20446ba442a6d9daf70371177e: Status 404 returned error can't find the container with id a872df7cb2015d7d1c0dca0187e93c6cd6c58d20446ba442a6d9daf70371177e
Oct 30 18:36:35.006762 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.006737    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hwxx2" event={"ID":"f98872b6-9c55-467c-9c70-2831de3fb841","Type":"ContainerStarted","Data":"9e4d6f461bec08d6e7ec3e1c131d33cb5a3240a2402b169bac3c75bdb0b6c255"}
Oct 30 18:36:35.007790 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.007767    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nr9vn" event={"ID":"9afdc202-0860-42d2-99dc-e1e028b75953","Type":"ContainerStarted","Data":"a872df7cb2015d7d1c0dca0187e93c6cd6c58d20446ba442a6d9daf70371177e"}
Oct 30 18:36:35.079621 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.079583    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b4fj5"]
Oct 30 18:36:35.083575 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:35.083551    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf51455e1_dbba_4186_adbf_60f44c9edbe5.slice/crio-cf3f35db32a715abe6d6fb9ec14c22d8e0bb478facd34ccc963be24ac3d0df58 WatchSource:0}: Error finding container cf3f35db32a715abe6d6fb9ec14c22d8e0bb478facd34ccc963be24ac3d0df58: Status 404 returned error can't find the container with id cf3f35db32a715abe6d6fb9ec14c22d8e0bb478facd34ccc963be24ac3d0df58
Oct 30 18:36:35.113670 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.113593    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hwxx2" podStartSLOduration=2.075952134 podStartE2EDuration="5.113579758s" totalImagesPullingTime="3.037627624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:35.112530127 +0000 UTC m=+34.893014774" watchObservedRunningTime="2026-10-30 18:36:35.113579758 +0000 UTC m=+34.894064407"
Oct 30 18:36:35.124369 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.124341    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qk8z2"]
Oct 30 18:36:35.128342 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:35.128318    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3e9afad_06d1_42eb_b463_5dc5932cd45d.slice/crio-b5e430f01d6ae61d520d3977d629b37e08265258a9b1949032de9568b1f0dc51 WatchSource:0}: Error finding container b5e430f01d6ae61d520d3977d629b37e08265258a9b1949032de9568b1f0dc51: Status 404 returned error can't find the container with id b5e430f01d6ae61d520d3977d629b37e08265258a9b1949032de9568b1f0dc51
Oct 30 18:36:35.471878 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.471846    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6zrww"]
Oct 30 18:36:35.505254 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.505231    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:35.554934 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.554907    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 18:36:35.556229 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.556208    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6zrww"]
Oct 30 18:36:35.643510 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.643470    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 18:36:35.643510 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.643499    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 18:36:35.643724 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.643472    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 18:36:35.643724 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.643674    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-swngw\""
Oct 30 18:36:35.653870 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.653847    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:35.653983 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.653896    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/11db0a9a-1a94-430c-b395-6ea6bbeba528-crio-socket\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:35.653983 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.653914    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqfr4\" (UniqueName: \"kubernetes.io/projected/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-api-access-zqfr4\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:35.654065 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.654048    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/11db0a9a-1a94-430c-b395-6ea6bbeba528-data-volume\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:35.654108 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.654095    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:35.696276 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.696253    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-pntg4-8mc75"]
Oct 30 18:36:35.715374 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.715355    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:35.752841 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.752790    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-pntg4-8mc75"]
Oct 30 18:36:35.755023 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.755001    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:35.755120 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.755056    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/11db0a9a-1a94-430c-b395-6ea6bbeba528-crio-socket\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:35.755120 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.755084    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqfr4\" (UniqueName: \"kubernetes.io/projected/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-api-access-zqfr4\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:35.755218 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.755123    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/11db0a9a-1a94-430c-b395-6ea6bbeba528-data-volume\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:35.755218 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:35.755160    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:35.755318 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:35.755241    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:36:35.755318 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.755161    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:35.755467 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.755322    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/11db0a9a-1a94-430c-b395-6ea6bbeba528-crio-socket\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:35.755467 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:35.755242    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls podName:11db0a9a-1a94-430c-b395-6ea6bbeba528 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:36.255224243 +0000 UTC m=+36.035708887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6zrww" (UID: "11db0a9a-1a94-430c-b395-6ea6bbeba528") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:35.755467 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:35.755358    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm podName:11db0a9a-1a94-430c-b395-6ea6bbeba528 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:36.25534457 +0000 UTC m=+36.035829207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6zrww" (UID: "11db0a9a-1a94-430c-b395-6ea6bbeba528") : configmap "kube-rbac-proxy" not found
Oct 30 18:36:35.755668 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.755562    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/11db0a9a-1a94-430c-b395-6ea6bbeba528-data-volume\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:35.764213 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.764194    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-xg84v\""
Oct 30 18:36:35.808710 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.808692    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqfr4\" (UniqueName: \"kubernetes.io/projected/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-api-access-zqfr4\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:35.843618 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.843561    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 18:36:35.855832 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.855811    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5cqd\" (UniqueName: \"kubernetes.io/projected/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-kube-api-access-p5cqd\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:35.855922 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.855843    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-service-ca-bundle\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:35.855922 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.855862    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-archives-path\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:35.855922 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.855894    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-trusted-ca-bundle\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:35.897500 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.897478    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 18:36:35.956566 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.956542    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5cqd\" (UniqueName: \"kubernetes.io/projected/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-kube-api-access-p5cqd\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:35.956688 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.956578    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-service-ca-bundle\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:35.956688 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.956613    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-archives-path\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:35.956688 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.956646    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-trusted-ca-bundle\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:35.957000 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.956978    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-archives-path\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:35.957257 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.957241    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-service-ca-bundle\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:35.957637 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:35.957621    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-trusted-ca-bundle\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:36.010193 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:36.010134    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5cqd\" (UniqueName: \"kubernetes.io/projected/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-kube-api-access-p5cqd\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:36.011194 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:36.011167    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b4fj5" event={"ID":"f51455e1-dbba-4186-adbf-60f44c9edbe5","Type":"ContainerStarted","Data":"cf3f35db32a715abe6d6fb9ec14c22d8e0bb478facd34ccc963be24ac3d0df58"}
Oct 30 18:36:36.012393 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:36.012372    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qk8z2" event={"ID":"f3e9afad-06d1-42eb-b463-5dc5932cd45d","Type":"ContainerStarted","Data":"b5e430f01d6ae61d520d3977d629b37e08265258a9b1949032de9568b1f0dc51"}
Oct 30 18:36:36.036303 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:36.036282    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:36.259526 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:36.259492    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:36.259702 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:36.259554    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:36.259702 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:36.259615    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:36:36.259702 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:36.259687    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:36.259859 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:36.259694    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm podName:11db0a9a-1a94-430c-b395-6ea6bbeba528 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:37.259675513 +0000 UTC m=+37.040160155 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6zrww" (UID: "11db0a9a-1a94-430c-b395-6ea6bbeba528") : configmap "kube-rbac-proxy" not found
Oct 30 18:36:36.259859 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:36.259761    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls podName:11db0a9a-1a94-430c-b395-6ea6bbeba528 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:37.259745541 +0000 UTC m=+37.040230175 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6zrww" (UID: "11db0a9a-1a94-430c-b395-6ea6bbeba528") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:36.412082 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:36.412044    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-pntg4-8mc75"]
Oct 30 18:36:36.418731 ip-10-0-140-122 kubenswrapper[2581]: W1030 18:36:36.418697    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda98b7c0d_2a11_4f58_ab3c_76e8bb2ae3c3.slice/crio-f77f8483d1e74bc955cbbc2ba67ad372126d051afb298b5f24534ac009886dd7 WatchSource:0}: Error finding container f77f8483d1e74bc955cbbc2ba67ad372126d051afb298b5f24534ac009886dd7: Status 404 returned error can't find the container with id f77f8483d1e74bc955cbbc2ba67ad372126d051afb298b5f24534ac009886dd7
Oct 30 18:36:37.016524 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:37.016488    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-pntg4-8mc75" event={"ID":"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3","Type":"ContainerStarted","Data":"f77f8483d1e74bc955cbbc2ba67ad372126d051afb298b5f24534ac009886dd7"}
Oct 30 18:36:37.267627 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:37.267547    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:37.267723 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:37.267663    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:37.267723 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:37.267706    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:37.267824 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:37.267749    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:36:37.267824 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:37.267772    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls podName:11db0a9a-1a94-430c-b395-6ea6bbeba528 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:39.267753418 +0000 UTC m=+39.048238059 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6zrww" (UID: "11db0a9a-1a94-430c-b395-6ea6bbeba528") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:37.267824 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:37.267796    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm podName:11db0a9a-1a94-430c-b395-6ea6bbeba528 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:39.267781124 +0000 UTC m=+39.048265765 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6zrww" (UID: "11db0a9a-1a94-430c-b395-6ea6bbeba528") : configmap "kube-rbac-proxy" not found
Oct 30 18:36:38.022537 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:38.022486    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nr9vn" event={"ID":"9afdc202-0860-42d2-99dc-e1e028b75953","Type":"ContainerStarted","Data":"ea09af7afc914c587d9c95298cb10956d2b3d60334a78abd7194cc82a2ab86b9"}
Oct 30 18:36:38.131833 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:38.131776    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nr9vn" podStartSLOduration=1.878308157 podStartE2EDuration="4.131759844s" totalImagesPullingTime="2.253451687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:38.130511129 +0000 UTC m=+37.910995780" watchObservedRunningTime="2026-10-30 18:36:38.131759844 +0000 UTC m=+37.912244492"
Oct 30 18:36:38.377061 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:38.376956    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:38.377231 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:38.377158    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:36:38.377231 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:38.377227    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls podName:5db1f514-bc37-424a-89ce-71dbc6a520e9 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:46.377206235 +0000 UTC m=+46.157690862 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls") pod "dns-default-zmj6x" (UID: "5db1f514-bc37-424a-89ce-71dbc6a520e9") : secret "dns-default-metrics-tls" not found
Oct 30 18:36:39.283991 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:39.283191    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:39.283991 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:39.283270    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:39.283991 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:39.283448    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:39.283991 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:39.283557    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:36:39.283991 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:39.283640    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm podName:11db0a9a-1a94-430c-b395-6ea6bbeba528 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:43.283617934 +0000 UTC m=+43.064102584 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6zrww" (UID: "11db0a9a-1a94-430c-b395-6ea6bbeba528") : configmap "kube-rbac-proxy" not found
Oct 30 18:36:39.284808 ip-10-0-140-122 kubenswrapper[2581]: E1030 18:36:39.284773    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls podName:11db0a9a-1a94-430c-b395-6ea6bbeba528 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:43.284752392 +0000 UTC m=+43.065237022 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6zrww" (UID: "11db0a9a-1a94-430c-b395-6ea6bbeba528") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:40.028949 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:40.028917    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qk8z2" event={"ID":"f3e9afad-06d1-42eb-b463-5dc5932cd45d","Type":"ContainerStarted","Data":"6ef7c4168c753d45e895d4299c0bfdfbaedcf4c3627eb935872d9c58fd7034f8"}
Oct 30 18:36:40.029147 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:40.029035    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:40.136708 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:40.136658    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qk8z2" podStartSLOduration=34.582089391 podStartE2EDuration="39.136637688s" totalImagesPullingTime="4.554548297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:40.135332047 +0000 UTC m=+39.915816697" watchObservedRunningTime="2026-10-30 18:36:40.136637688 +0000 UTC m=+39.917122337"
Oct 30 18:36:41.033352 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:41.033313    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-pntg4-8mc75" event={"ID":"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3","Type":"ContainerStarted","Data":"f4ce3e1ab0b44b64afcdeb40f40b6839ebbd4563df941c82da0a4e0a4380e6b1"}
Oct 30 18:36:41.143734 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:41.141815    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-pntg4-8mc75" podStartSLOduration=2.477654845 podStartE2EDuration="6.141799274s" totalImagesPullingTime="3.664144429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:41.140289412 +0000 UTC m=+40.920774062" watchObservedRunningTime="2026-10-30 18:36:41.141799274 +0000 UTC m=+40.922283921"
Oct 30 18:36:42.397913 ip-10-0-140-122 kubenswrapper[2581]: I1030 18:36:42.397705    2581 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:36:42.397877 ip-10-0-140-122 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 18:36:42.417005 ip-10-0-140-122 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 18:36:42.417220 ip-10-0-140-122 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 18:36:42.417423 ip-10-0-140-122 systemd[1]: kubelet.service: Consumed 2.317s CPU time.
Oct 30 18:36:42.428153 ip-10-0-140-122 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 18:36:42.512383 ip-10-0-140-122 kubenswrapper[4544]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:36:42.512383 ip-10-0-140-122 kubenswrapper[4544]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 18:36:42.512383 ip-10-0-140-122 kubenswrapper[4544]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:36:42.512383 ip-10-0-140-122 kubenswrapper[4544]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:36:42.514413 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514336    4544 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:42.514413 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514413    4544 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514417    4544 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514421    4544 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514424    4544 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514427    4544 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514430    4544 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514433    4544 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514436    4544 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514439    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514442    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514444    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514447    4544 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514450    4544 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514453    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514456    4544 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514458    4544 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514461    4544 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514464    4544 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514466    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514469    4544 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:42.514484 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514477    4544 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514481    4544 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514484    4544 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514487    4544 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514490    4544 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514493    4544 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514497    4544 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514500    4544 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514502    4544 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514505    4544 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514508    4544 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514511    4544 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514515    4544 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514519    4544 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514522    4544 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514525    4544 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514528    4544 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514530    4544 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514533    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514536    4544 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:42.515003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514538    4544 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514541    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514543    4544 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514546    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514549    4544 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514551    4544 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514554    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514557    4544 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514559    4544 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514562    4544 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514564    4544 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514567    4544 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514570    4544 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514573    4544 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514576    4544 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514579    4544 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514582    4544 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514584    4544 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514587    4544 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514589    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:42.515510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514592    4544 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514595    4544 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514598    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514612    4544 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514615    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514617    4544 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514620    4544 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514622    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514625    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514628    4544 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514631    4544 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514634    4544 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514637    4544 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514641    4544 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514645    4544 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514648    4544 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514651    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514654    4544 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514657    4544 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:42.516031 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514659    4544 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514662    4544 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514664    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514667    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514670    4544 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514672    4544 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514675    4544 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514679    4544 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514681    4544 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514684    4544 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514687    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514690    4544 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514692    4544 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514695    4544 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514697    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514700    4544 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514702    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514705    4544 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514707    4544 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:42.516506 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514711    4544 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514714    4544 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514716    4544 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514719    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514722    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514724    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.514728    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515441    4544 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515447    4544 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515450    4544 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515452    4544 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515455    4544 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515458    4544 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515461    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515463    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515466    4544 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515469    4544 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515472    4544 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515474    4544 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:42.516994 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515485    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515488    4544 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515491    4544 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515494    4544 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515497    4544 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515499    4544 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515502    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515504    4544 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515507    4544 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515510    4544 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515513    4544 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515516    4544 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515519    4544 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515522    4544 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515525    4544 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515527    4544 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515529    4544 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515532    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515535    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515537    4544 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:42.517477 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515541    4544 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515543    4544 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515546    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515548    4544 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515551    4544 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515554    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515557    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515561    4544 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515564    4544 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515567    4544 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515570    4544 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515572    4544 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515575    4544 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515577    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515580    4544 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515582    4544 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515584    4544 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515588    4544 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515590    4544 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:42.518028 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515593    4544 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515595    4544 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515598    4544 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515616    4544 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515620    4544 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515625    4544 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515629    4544 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515634    4544 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515640    4544 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515643    4544 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515646    4544 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515650    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515653    4544 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515656    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515659    4544 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515662    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515665    4544 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515667    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515670    4544 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:42.518517 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515673    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515675    4544 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515678    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515680    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515683    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515685    4544 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515688    4544 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515690    4544 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515693    4544 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515696    4544 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515699    4544 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515701    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515704    4544 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515707    4544 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515709    4544 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515712    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515714    4544 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515717    4544 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515720    4544 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515722    4544 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:42.519039 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515725    4544 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515727    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515730    4544 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515733    4544 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515744    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515747    4544 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515749    4544 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515752    4544 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515755    4544 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515757    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515760    4544 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515762    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515765    4544 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515769    4544 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515772    4544 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.515775    4544 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516259    4544 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516265    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516267    4544 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516271    4544 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:42.519540 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516274    4544 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516280    4544 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516283    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516286    4544 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516289    4544 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516292    4544 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516294    4544 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516297    4544 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516299    4544 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516302    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516305    4544 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516309    4544 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516311    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516314    4544 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516317    4544 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516320    4544 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516322    4544 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516325    4544 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516327    4544 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516330    4544 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:42.520071 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516333    4544 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516336    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516338    4544 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516341    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516344    4544 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516346    4544 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516349    4544 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516351    4544 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516354    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516356    4544 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516359    4544 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516361    4544 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516364    4544 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516368    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516372    4544 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516375    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516377    4544 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516380    4544 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516382    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:42.520949 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516385    4544 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516387    4544 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516390    4544 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516393    4544 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516395    4544 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516398    4544 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516401    4544 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516403    4544 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516406    4544 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516408    4544 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516411    4544 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516414    4544 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516418    4544 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516421    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516425    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516428    4544 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516433    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516436    4544 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516439    4544 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:42.521802 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516441    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516444    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516446    4544 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516449    4544 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516451    4544 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516454    4544 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516456    4544 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516460    4544 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516463    4544 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516466    4544 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516468    4544 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516470    4544 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516473    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516476    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516478    4544 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516481    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516483    4544 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516486    4544 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516489    4544 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516491    4544 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:42.522652 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516494    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516496    4544 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516499    4544 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516501    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516504    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516507    4544 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516509    4544 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516512    4544 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516515    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516518    4544 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516521    4544 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516523    4544 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516526    4544 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516529    4544 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516532    4544 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516534    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516537    4544 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516539    4544 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516542    4544 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:42.523562 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516545    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516548    4544 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516550    4544 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516553    4544 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.516555    4544 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516665    4544 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516683    4544 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516694    4544 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516699    4544 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516703    4544 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516706    4544 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516711    4544 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516715    4544 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516718    4544 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516721    4544 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516725    4544 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516728    4544 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516731    4544 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516734    4544 flags.go:64] FLAG: --cgroup-root=""
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516737    4544 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516740    4544 flags.go:64] FLAG: --client-ca-file=""
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516743    4544 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516752    4544 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 18:36:42.524421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516761    4544 flags.go:64] FLAG: --cluster-domain=""
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516764    4544 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516767    4544 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516770    4544 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516773    4544 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516778    4544 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516783    4544 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516786    4544 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516789    4544 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516792    4544 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516796    4544 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516798    4544 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516802    4544 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516805    4544 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516809    4544 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516812    4544 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516815    4544 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516818    4544 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516821    4544 flags.go:64] FLAG: --enable-server="true"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516824    4544 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516831    4544 flags.go:64] FLAG: --event-burst="100"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516834    4544 flags.go:64] FLAG: --event-qps="50"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516837    4544 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516840    4544 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 18:36:42.525299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516843    4544 flags.go:64] FLAG: --eviction-hard=""
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516847    4544 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516851    4544 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516854    4544 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516858    4544 flags.go:64] FLAG: --eviction-soft=""
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516861    4544 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516864    4544 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516867    4544 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516870    4544 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516873    4544 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516876    4544 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516879    4544 flags.go:64] FLAG: --feature-gates=""
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516883    4544 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516887    4544 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516892    4544 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516896    4544 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516899    4544 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516902    4544 flags.go:64] FLAG: --help="false"
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516905    4544 flags.go:64] FLAG: --hostname-override="ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516908    4544 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516911    4544 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516914    4544 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516918    4544 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 18:36:42.526324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516921    4544 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516924    4544 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516927    4544 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516930    4544 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516932    4544 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516935    4544 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516938    4544 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516941    4544 flags.go:64] FLAG: --kube-reserved=""
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516944    4544 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516947    4544 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516950    4544 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516953    4544 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516956    4544 flags.go:64] FLAG: --lock-file=""
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516959    4544 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516962    4544 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516965    4544 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516971    4544 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516973    4544 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516977    4544 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516980    4544 flags.go:64] FLAG: --logging-format="text"
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516983    4544 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516986    4544 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516989    4544 flags.go:64] FLAG: --manifest-url=""
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516993    4544 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.516998    4544 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 18:36:42.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517001    4544 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517005    4544 flags.go:64] FLAG: --max-pods="110"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517008    4544 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517011    4544 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517014    4544 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517016    4544 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517019    4544 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517022    4544 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517025    4544 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517032    4544 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517035    4544 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517038    4544 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517041    4544 flags.go:64] FLAG: --pod-cidr=""
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517045    4544 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517048    4544 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517052    4544 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517055    4544 flags.go:64] FLAG: --port="10250"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517058    4544 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517061    4544 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-06aea14225dec2643"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517064    4544 flags.go:64] FLAG: --qos-reserved=""
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517067    4544 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517070    4544 flags.go:64] FLAG: --register-node="true"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517073    4544 flags.go:64] FLAG: --register-with-taints=""
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517077    4544 flags.go:64] FLAG: --registry-burst="10"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517080    4544 flags.go:64] FLAG: --registry-qps="5"
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517082    4544 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 18:36:42.527586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517089    4544 flags.go:64] FLAG: --reserved-memory=""
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517093    4544 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517096    4544 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517099    4544 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517104    4544 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517107    4544 flags.go:64] FLAG: --runonce="false"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517110    4544 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517113    4544 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517116    4544 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517119    4544 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517122    4544 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517125    4544 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517128    4544 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517131    4544 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517134    4544 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517137    4544 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517140    4544 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517143    4544 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517146    4544 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517150    4544 flags.go:64] FLAG: --system-cgroups=""
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517153    4544 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517158    4544 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517161    4544 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517164    4544 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517170    4544 flags.go:64] FLAG: --tls-min-version=""
Oct 30 18:36:42.528370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517173    4544 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517176    4544 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517179    4544 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517182    4544 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517185    4544 flags.go:64] FLAG: --v="2"
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517190    4544 flags.go:64] FLAG: --version="false"
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517194    4544 flags.go:64] FLAG: --vmodule=""
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517198    4544 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517207    4544 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517360    4544 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517364    4544 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517369    4544 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517372    4544 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517374    4544 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517377    4544 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517380    4544 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517382    4544 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517385    4544 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517388    4544 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517390    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517393    4544 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:42.528998 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517395    4544 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517398    4544 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517400    4544 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517403    4544 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517405    4544 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517409    4544 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517412    4544 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517415    4544 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517418    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517421    4544 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517423    4544 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517426    4544 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517429    4544 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517432    4544 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517436    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517438    4544 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517441    4544 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517444    4544 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517446    4544 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:42.529507 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517449    4544 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517452    4544 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517454    4544 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517458    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517461    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517463    4544 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517466    4544 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517469    4544 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517471    4544 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517474    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517476    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517479    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517482    4544 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517484    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517487    4544 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517490    4544 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517493    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517496    4544 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517500    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:42.530063 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517502    4544 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517505    4544 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517507    4544 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517512    4544 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517514    4544 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517517    4544 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517520    4544 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517523    4544 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517525    4544 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517528    4544 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517530    4544 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517533    4544 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517536    4544 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517538    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517541    4544 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517544    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517548    4544 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517551    4544 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517553    4544 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517556    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:42.530669 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517558    4544 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517561    4544 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517564    4544 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517566    4544 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517569    4544 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517572    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517574    4544 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517577    4544 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517579    4544 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517582    4544 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517584    4544 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517588    4544 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517591    4544 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517593    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517596    4544 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517612    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517615    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517618    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517620    4544 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517623    4544 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:42.531315 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517626    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:42.531907 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517628    4544 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:42.531907 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517631    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:42.531907 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517633    4544 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:42.531907 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517636    4544 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:42.531907 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517639    4544 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:42.531907 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517641    4544 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:42.531907 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517644    4544 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:42.531907 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517648    4544 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:42.531907 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517651    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:42.531907 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517654    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:42.531907 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517656    4544 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:42.531907 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517659    4544 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:42.531907 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517661    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:42.531907 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517665    4544 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:42.531907 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517669    4544 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:42.531907 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.517697    4544 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517734    4544 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517739    4544 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517743    4544 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517746    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517749    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517751    4544 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517754    4544 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517758    4544 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517761    4544 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517763    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517766    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517768    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517771    4544 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517773    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517776    4544 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517779    4544 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517781    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517784    4544 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:42.532497 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517786    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517789    4544 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517791    4544 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517794    4544 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517797    4544 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517801    4544 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517804    4544 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517807    4544 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517809    4544 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517812    4544 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517814    4544 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517817    4544 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517820    4544 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517822    4544 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517825    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517827    4544 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517830    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517833    4544 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517835    4544 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517838    4544 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:42.533053 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517840    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517844    4544 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517847    4544 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517850    4544 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517852    4544 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517855    4544 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517858    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517860    4544 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517863    4544 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517866    4544 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517868    4544 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517871    4544 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517873    4544 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517876    4544 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517879    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517881    4544 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517884    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517888    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517891    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517894    4544 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:42.533580 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517896    4544 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517899    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517902    4544 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517904    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517907    4544 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517910    4544 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517912    4544 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517915    4544 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517917    4544 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517920    4544 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517922    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517925    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517927    4544 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517931    4544 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517934    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517939    4544 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517943    4544 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517946    4544 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517949    4544 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:42.534083 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517952    4544 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517955    4544 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517957    4544 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517960    4544 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517963    4544 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517965    4544 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517968    4544 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517970    4544 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517973    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517975    4544 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517979    4544 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517982    4544 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517984    4544 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517987    4544 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517990    4544 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517993    4544 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517995    4544 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.517998    4544 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.518000    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.518003    4544 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:42.534575 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.518006    4544 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.518008    4544 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.518011    4544 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.518015    4544 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.518019    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.518022    4544 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.518026    4544 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.518029    4544 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.518032    4544 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.522770    4544 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.522788    4544 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.522812    4544 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.522820    4544 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.522946    4544 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.522954    4544 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.522958    4544 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.522964    4544 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.522973    4544 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.522977    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:42.535094 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.522982    4544 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.522987    4544 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.522991    4544 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.522996    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523000    4544 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523004    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523017    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523021    4544 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523026    4544 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523030    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523034    4544 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523038    4544 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523043    4544 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523056    4544 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523060    4544 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523065    4544 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523069    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523075    4544 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523079    4544 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523083    4544 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:42.535573 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523088    4544 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523092    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523096    4544 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523101    4544 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523105    4544 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523109    4544 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523114    4544 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523118    4544 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523123    4544 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523129    4544 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523136    4544 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523142    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523146    4544 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523151    4544 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523155    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523159    4544 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523164    4544 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523168    4544 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523172    4544 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:42.536087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523184    4544 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523189    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523193    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523197    4544 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523201    4544 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523205    4544 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523210    4544 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523214    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523219    4544 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523224    4544 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523229    4544 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523233    4544 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523238    4544 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523242    4544 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523247    4544 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523251    4544 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523256    4544 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523260    4544 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523264    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:42.536554 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523269    4544 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523273    4544 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523278    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523282    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523286    4544 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523291    4544 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523295    4544 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523300    4544 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523304    4544 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523308    4544 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523312    4544 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523317    4544 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523321    4544 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523325    4544 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523337    4544 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523342    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523347    4544 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523351    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523355    4544 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523360    4544 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:42.537029 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523364    4544 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523368    4544 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523372    4544 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523377    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523382    4544 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523387    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523391    4544 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523395    4544 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523399    4544 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523404    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523408    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523412    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523416    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523420    4544 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523424    4544 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523429    4544 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523433    4544 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523438    4544 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523442    4544 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:42.537510 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523449    4544 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:42.538003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523456    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:42.538003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523461    4544 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:42.538003 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.523511    4544 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:36:42.538003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523830    4544 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:42.538003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523841    4544 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:42.538003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523846    4544 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:42.538003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523851    4544 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:42.538003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523863    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:42.538003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523868    4544 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:42.538003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523872    4544 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:42.538003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523877    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:42.538003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523881    4544 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:42.538003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523885    4544 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:42.538003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523890    4544 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:42.538003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523895    4544 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:42.538003 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523899    4544 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523904    4544 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523908    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523913    4544 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523917    4544 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523922    4544 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523926    4544 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523930    4544 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523935    4544 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523939    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523943    4544 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523947    4544 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523951    4544 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523956    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523960    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523964    4544 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523968    4544 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523973    4544 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523977    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:42.538381 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523982    4544 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523986    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523990    4544 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523994    4544 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.523998    4544 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524003    4544 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524015    4544 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524019    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524024    4544 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524028    4544 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524032    4544 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524037    4544 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524041    4544 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524045    4544 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524050    4544 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524054    4544 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524059    4544 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524063    4544 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524067    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524072    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:42.538931 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524076    4544 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524080    4544 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524085    4544 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524089    4544 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524093    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524098    4544 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524102    4544 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524106    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524110    4544 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524114    4544 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524119    4544 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524123    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524128    4544 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524132    4544 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524136    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524141    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524145    4544 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524149    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524154    4544 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524165    4544 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:42.539512 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524170    4544 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524174    4544 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524178    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524182    4544 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524187    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524191    4544 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524196    4544 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524200    4544 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524204    4544 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524209    4544 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524213    4544 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524219    4544 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524227    4544 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524232    4544 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524238    4544 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524242    4544 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524247    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524252    4544 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524257    4544 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524262    4544 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:42.540254 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524266    4544 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:42.541032 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524270    4544 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:42.541032 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524275    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:42.541032 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524279    4544 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:42.541032 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524283    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:42.541032 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524287    4544 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:42.541032 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524292    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:42.541032 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524296    4544 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:42.541032 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524303    4544 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:42.541032 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524307    4544 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:42.541032 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524312    4544 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:42.541032 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524316    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:42.541032 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524330    4544 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:42.541032 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524334    4544 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:42.541032 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.524339    4544 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:42.541032 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.524379    4544 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525433    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525445    4544 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525451    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525455    4544 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525460    4544 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525465    4544 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525470    4544 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525480    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525485    4544 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525489    4544 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525494    4544 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525498    4544 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525503    4544 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525508    4544 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525512    4544 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525517    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525521    4544 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525526    4544 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:42.541412 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525530    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525535    4544 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525545    4544 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525550    4544 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525557    4544 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525561    4544 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525566    4544 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525571    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525576    4544 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525580    4544 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525585    4544 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525661    4544 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525668    4544 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525673    4544 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525683    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525687    4544 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525691    4544 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525696    4544 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525700    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525705    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:42.542069 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525709    4544 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525714    4544 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525718    4544 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525722    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525727    4544 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525731    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525737    4544 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525816    4544 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525836    4544 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525842    4544 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525847    4544 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525952    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525960    4544 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525963    4544 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525965    4544 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525968    4544 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525971    4544 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525974    4544 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525976    4544 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:42.543474 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525983    4544 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525986    4544 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525988    4544 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525991    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525994    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525996    4544 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.525999    4544 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526002    4544 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526008    4544 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526012    4544 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526015    4544 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526017    4544 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526020    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526022    4544 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526025    4544 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526028    4544 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526032    4544 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526035    4544 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526038    4544 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526040    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:42.544279 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526043    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526045    4544 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526048    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526051    4544 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526054    4544 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526056    4544 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526059    4544 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526064    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526066    4544 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526071    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526074    4544 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526076    4544 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526081    4544 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526084    4544 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526087    4544 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526089    4544 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526094    4544 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526098    4544 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526104    4544 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:42.545087 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526108    4544 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:42.545750 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526111    4544 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:42.545750 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526114    4544 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:42.545750 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526117    4544 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:42.545750 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526119    4544 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:42.545750 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526122    4544 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:42.545750 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526125    4544 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:42.545750 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526127    4544 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:42.545750 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526130    4544 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:42.545750 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:42.526136    4544 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:42.545750 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.526219    4544 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 18:36:42.545750 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.527199    4544 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 18:36:42.545750 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.527255    4544 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 18:36:42.545750 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.527426    4544 server.go:1014] "Starting client certificate rotation"
Oct 30 18:36:42.545750 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.527578    4544 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:36:42.545750 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.527653    4544 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:31:01 +0000 UTC" deadline="2028-03-27 10:35:02.106220607 +0000 UTC"
Oct 30 18:36:42.545750 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.527697    4544 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12327h58m19.578527133s"
Oct 30 18:36:42.546162 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.527983    4544 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:36:42.546162 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.529092    4544 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:36:42.546162 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.530443    4544 log.go:25] "Validated CRI v1 runtime API"
Oct 30 18:36:42.546162 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.532157    4544 log.go:25] "Validated CRI v1 image API"
Oct 30 18:36:42.546162 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.532936    4544 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 18:36:42.546162 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.534447    4544 fs.go:135] Filesystem UUIDs: map[6e6dd817-e5ce-4d37-8e08-f6c04abfe3d9:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 80bc89d5-3804-4b53-8857-1554ffa4328d:/dev/nvme0n1p3]
Oct 30 18:36:42.546357 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.534470    4544 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/197507b0c77d583cadd2a8de2b1452d42ccb42e153c5d06b52cde51bb4a195e6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/197507b0c77d583cadd2a8de2b1452d42ccb42e153c5d06b52cde51bb4a195e6/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/36d94e2489c4d23d3321b3cf5a02717ffbd01c4d39715c1e26d42eaf340cdbdc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/36d94e2489c4d23d3321b3cf5a02717ffbd01c4d39715c1e26d42eaf340cdbdc/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3da38b7898a74e5db9cfbea383d4ba1963ac2528558e42bac58423fd58d14ac0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3da38b7898a74e5db9cfbea383d4ba1963ac2528558e42bac58423fd58d14ac0/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3ebae1f01b86ba4f27b38d27f5f5b48f9d53d221f8cf0665322b56c1e4591869/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3ebae1f01b86ba4f27b38d27f5f5b48f9d53d221f8cf0665322b56c1e4591869/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/74d238c6a8c7ac178725be85164bcec19e4f1316bab30352419cc161f7577d43/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/74d238c6a8c7ac178725be85164bcec19e4f1316bab30352419cc161f7577d43/userdata/shm major:0 minor:79 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7cbbb3a060e45b04b442cd03dd28b9df91f2132707a0bcf475d14a91a6ae994b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7cbbb3a060e45b04b442cd03dd28b9df91f2132707a0bcf475d14a91a6ae994b/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9156ceacadc3ef5c17ff0a82459481ea6c940b128e65c276ddb10d0106554d53/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9156ceacadc3ef5c17ff0a82459481ea6c940b128e65c276ddb10d0106554d53/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9963f1102736ee5cf0c733cf42c99fd2f2cec60d380ed11e1d351fed5622cbce/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9963f1102736ee5cf0c733cf42c99fd2f2cec60d380ed11e1d351fed5622cbce/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9e8c259aaafc99b31cb10786e94629f2628ff0a257e257b79dd1858155e43d55/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9e8c259aaafc99b31cb10786e94629f2628ff0a257e257b79dd1858155e43d55/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a872df7cb2015d7d1c0dca0187e93c6cd6c58d20446ba442a6d9daf70371177e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a872df7cb2015d7d1c0dca0187e93c6cd6c58d20446ba442a6d9daf70371177e/userdata/shm major:0 minor:215 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b5e430f01d6ae61d520d3977d629b37e08265258a9b1949032de9568b1f0dc51/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b5e430f01d6ae61d520d3977d629b37e08265258a9b1949032de9568b1f0dc51/userdata/shm major:0 minor:219 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cf3f35db32a715abe6d6fb9ec14c22d8e0bb478facd34ccc963be24ac3d0df58/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cf3f35db32a715abe6d6fb9ec14c22d8e0bb478facd34ccc963be24ac3d0df58/userdata/shm major:0 minor:217 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/de9b062be5610c482edd33d85e74f7bf16a925f7219417dbf1e2c6e9a9b03485/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/de9b062be5610c482edd33d85e74f7bf16a925f7219417dbf1e2c6e9a9b03485/userdata/shm major:0 minor:93 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e7dce758fa820d22634e473f8ac42aef98a5118f84adc773ff1426d941333629/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e7dce758fa820d22634e473f8ac42aef98a5118f84adc773ff1426d941333629/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f77f8483d1e74bc955cbbc2ba67ad372126d051afb298b5f24534ac009886dd7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f77f8483d1e74bc955cbbc2ba67ad372126d051afb298b5f24534ac009886dd7/userdata/shm major:0 minor:229 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/114fbcb8-1004-410d-98bc-dc60f8fcae3c/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/114fbcb8-1004-410d-98bc-dc60f8fcae3c/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/114fbcb8-1004-410d-98bc-dc60f8fcae3c/volumes/kubernetes.io~projected/kube-api-access-pphxn:{mountpoint:/var/lib/kubelet/pods/114fbcb8-1004-410d-98bc-dc60f8fcae3c/volumes/kubernetes.io~projected/kube-api-access-pphxn major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/114fbcb8-1004-410d-98bc-dc60f8fcae3c/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/114fbcb8-1004-410d-98bc-dc60f8fcae3c/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/11db0a9a-1a94-430c-b395-6ea6bbeba528/volumes/kubernetes.io~projected/kube-api-access-zqfr4:{mountpoint:/var/lib/kubelet/pods/11db0a9a-1a94-430c-b395-6ea6bbeba528/volumes/kubernetes.io~projected/kube-api-access-zqfr4 major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/299ac5c9-51d1-4903-b7f1-2001aa7b2bcf/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/299ac5c9-51d1-4903-b7f1-2001aa7b2bcf/volumes/kubernetes.io~secret/agent-certs major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a6021d6-6317-473f-97bc-ea023261d90b/volumes/kubernetes.io~projected/kube-api-access-npfv7:{mountpoint:/var/lib/kubelet/pods/4a6021d6-6317-473f-97bc-ea023261d90b/volumes/kubernetes.io~projected/kube-api-access-npfv7 major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5db1f514-bc37-424a-89ce-71dbc6a520e9/volumes/kubernetes.io~projected/kube-api-access-j82vx:{mountpoint:/var/lib/kubelet/pods/5db1f514-bc37-424a-89ce-71dbc6a520e9/volumes/kubernetes.io~projected/kube-api-access-j82vx major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/90c6593d-b40a-43ae-abc0-ce015e48c289/volumes/kubernetes.io~projected/kube-api-access-8clvh:{mountpoint:/var/lib/kubelet/pods/90c6593d-b40a-43ae-abc0-ce015e48c289/volumes/kubernetes.io~projected/kube-api-access-8clvh major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/92a21826-d145-4b9b-9897-aaf21f63a67b/volumes/kubernetes.io~projected/kube-api-access-c2fqp:{mountpoint:/var/lib/kubelet/pods/92a21826-d145-4b9b-9897-aaf21f63a67b/volumes/kubernetes.io~projected/kube-api-access-c2fqp major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9afdc202-0860-42d2-99dc-e1e028b75953/volumes/kubernetes.io~projected/kube-api-access-sz7ff:{mountpoint:/var/lib/kubelet/pods/9afdc202-0860-42d2-99dc-e1e028b75953/volumes/kubernetes.io~projected/kube-api-access-sz7ff major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a517bba6-536f-4dac-ba8b-2e71964ef8c5/volumes/kubernetes.io~projected/kube-api-access-vnjp7:{mountpoint:/var/lib/kubelet/pods/a517bba6-536f-4dac-ba8b-2e71964ef8c5/volumes/kubernetes.io~projected/kube-api-access-vnjp7 major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3/volumes/kubernetes.io~projected/kube-api-access-p5cqd:{mountpoint:/var/lib/kubelet/pods/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3/volumes/kubernetes.io~projected/kube-api-access-p5cqd major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ab0116fc-0400-4f7d-be91-27804b24c8f4/volumes/kubernetes.io~projected/kube-api-access-tgj6t:{mountpoint:/var/lib/kubelet/pods/ab0116fc-0400-4f7d-be91-27804b24c8f4/volumes/kubernetes.io~projected/kube-api-access-tgj6t major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d7ebad18-eccb-47c9-b259-67c6d6b84d0f/volumes/kubernetes.io~projected/kube-api-access-2hwhq:{mountpoint:/var/lib/kubelet/pods/d7ebad18-eccb-47c9-b259-67c6d6b84d0f/volumes/kubernetes.io~projected/kube-api-access-2hwhq major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d93ec858-ae33-473a-ad4b-18b88664094c/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/d93ec858-ae33-473a-ad4b-18b88664094c/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d93ec858-ae33-473a-ad4b-18b88664094c/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/d93ec858-ae33-473a-ad4b-18b88664094c/volumes/kubernetes.io~empty-dir/tmp major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d93ec858-ae33-473a-ad4b-18b88664094c/volumes/kubernetes.io~projected/kube-api-access-dhx9h:{mountpoint:/var/lib/kubelet/pods/d93ec858-ae33-473a-ad4b-18b88664094c/volumes/kubernetes.io~projected/kube-api-access-dhx9h major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f3e9afad-06d1-42eb-b463-5dc5932cd45d/volumes/kubernetes.io~projected/kube-api-access-fhn6w:{mountpoint:/var/lib/kubelet/pods/f3e9afad-06d1-42eb-b463-5dc5932cd45d/volumes/kubernetes.io~projected/kube-api-access-fhn6w major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f51455e1-dbba-4186-adbf-60f44c9edbe5/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/f51455e1-dbba-4186-adbf-60f44c9edbe5/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f7894bc9-1a77-42e3-a59b-926729c93dc9/volumes/kubernetes.io~projected/kube-api-access-bdsjl:{mountpoint:/var/lib/kubelet/pods/f7894bc9-1a77-42e3-a59b-926729c93dc9/volumes/kubernetes.io~projected/kube-api-access-bdsjl major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f98872b6-9c55-467c-9c70-2831de3fb841/volumes/kubernetes.io~projected/kube-api-access-vn988:{mountpoint:/var/lib/kubelet/pods/f98872b6-9c55-467c-9c70-2831de3fb841/volumes/kubernetes.io~projected/kube-api-access-vn988 major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f98872b6-9c55-467c-9c70-2831de3fb841/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/f98872b6-9c55-467c-9c70-2831de3fb841/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/307bbb391623302885f6e87f76438000d419cbbf3fdfb9653699c8ac665b1014/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/c5d805922b6c71f59fb5321ecbc07a7aac56b86700b50021149e29ed66856976/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/d90f75d6661e56ba79419988e1669fac89be078f6154b35ec81ecceb1bde5248/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/62a327078cff6024cf3380251fbf383d6de9864783e65da37d8fb5f8d3c9a0c6/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-108:{mountpoint:/var/lib/containers/storage/overlay/9ef7dfb8b071a88aefbae68bcd0818b3dd60dcc1e37cac811dcb55f68e3dafcf/merged major:0 minor:108 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/a812e30f46134a45b16814edd0f4d51cfb7c5a3a55b90f8b7e1b478873b4e917/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/2181cd210ba9e4e510af0e59fd3bc18ab2b970081637013d31d479acde232ff4/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/d3f3ec0009294204c1cdbd6b55931ec10bdd557b763e4c085457b0f6583fea9b/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/d5925b840f8548af6d8bc437e5f90e41ee972ac31168dcc1511f836b1c724f04/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/14b0934367d9fe01d6185b46eb0a9d5da6bec04e200db9efdd268719fed379f1/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/e6d46959535fdb076bfa17a590bd6b77de49d44cabed5b8e1883b0b540586d7a/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/a10ff0ce682870b942b429ea34628dc6cf1228f51a0554613db081bcc923eedd/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/95d45a3707d531ad3f7763c64f9e03c260a0bd3f00ac31d117181e9bf843ebfd/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-154:{mountpoint:/var/lib/containers/storage/overlay/e9897447a2953954ab80ebb185282c1c3ca535ae29c1a5245bf933f2ccfdfdc1/merged major:0 minor:154 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/f0b160b6d06d0057081266b96c4e07bdddb7f5ece008e80cb63908ac7736045b/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/ad97547de9b554d241a60481488fa2365b071036ae78a55d6be771e236d0d338/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/6fbd279845d491426a17aa8fbb0397b6b4b8ad7ecaffb8139bcf71320a104727/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/35e2e6dceb8156405b65b33d78547139763f7351fbcc8eeb1137c41e5fd32d6a/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/9931cc8496ac41599621b3c05348c467373a764c8793069a9f57c19c048d791e/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/c0a6f543d973459b8ca6aa636d5d2d077116d6d776ac040992f94f490827e0aa/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/081a657aab270812ccf3244aa3d100af3af70499c4842cf4766e309da0a9e900/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-204:{mountpoint:/var/lib/containers/storage/overlay/a89f3cab77105607e6bfcc406bff63a17f1313234b24d732d26e1a2cea755799/merged major:0 minor:204 fsType:overlay blockSize:0} overlay_0-206:{mountpoint:/var/lib/containers/storage/overlay/6dc9da26153a37e3668a288283c898ab29e4d440cd7ce3cc200b6354b1996b6e/merged major:0 minor:206 fsType:overlay blockSize:0} overlay_0-221:{mountpoint:/var/lib/containers/storage/overlay/38bc402efdd29c135a243f72549ba1d23ba2d516a4c4a13a5e145ad40ce6262c/merged major:0 minor:221 fsType:overlay blockSize:0} overlay_0-223:{mountpoint:/var/lib/containers/storage/overlay/a729cc1890cca86f469af96b7e846585299a9f68008cf3f053ef21a1c1bd0e8a/merged major:0 minor:223 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/8b7408b4ed3870eda3e6ac3e74ecc85a3851aa1625a14be25e856dc2aad5228f/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-231:{mountpoint:/var/lib/containers/storage/overlay/525e2571c87e874b06ff5df6d9e03780e8aeea0ed7165130142bc42951fe5ccb/merged major:0 minor:231 fsType:overlay blockSize:0} overlay_0-233:{mountpoint:/var/lib/containers/storage/overlay/5958804681c3b3a1daaa1d847b390810be3445f111cededf8a6c62b52d09b867/merged major:0 minor:233 fsType:overlay blockSize:0} overlay_0-235:{mountpoint:/var/lib/containers/storage/overlay/c0620ce724147b45fcadbe2e54569dab3fedca2e2e321ba59218f113b39e0545/merged major:0 minor:235 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/bd9219f3fb2065b6d38fee7cef6390b5c2fcf3f4bf55a2038b285f085ae44e46/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/23d712ca3bf6c630ee085f4049e3fded130ffb02c676d4d05934ebf0b33eed9a/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/39c3b7194af3bdb73b1005f1d7fbd4f4369d463a5f52bcbb092947ac2c009c33/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/4e101e9d03df55dd591d42c2a78b8a11775f298335ce2d45415546f4c247f35a/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/5fc7e7bba19d8c1a427cdc200821409aad108e1136462df4468c7c6558d481a0/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/b58bd5cd5bc7511de5691ca189192e5b230639a298f4ab97a458d8053bd2173e/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/78d2fb9d8ace6707fe9820e67c7750fb9a2e863f4bb10f77add208d46825ed4f/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/fd305eb52818053d454e9af398a7d252fa502e4f3a36670dc13a5088e352d6de/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/c86a2a97c278b664b63723e22f0c74043522f2f4ce4a7948379cae1051cb844c/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/4a5da8f282a537ad1496ae1b170cc765f8316f5cb196b3e6043ba72e704db8ea/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/0139f554c370faa81016392ed8d5c56730c1feec9f82f674274d4769c14cea6c/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/50f30a47b50655da2df5ed5c326cb5693c252320cfea9492a9a09fee0ac0b510/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.541820    4544 manager.go:217] Machine: {Timestamp:2026-10-30 18:36:42.541077286 +0000 UTC m=+0.066952374 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3085088 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec204d1e3a779c06e5f9875099646a10 SystemUUID:ec204d1e-3a77-9c06-e5f9-875099646a10 BootID:f634fc99-5ea3-4a46-bf49-221d2d807091 Filesystems:[{Device:overlay_0-108 DeviceMajor:0 DeviceMinor:108 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-233 DeviceMajor:0 DeviceMinor:233 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f3e9afad-06d1-42eb-b463-5dc5932cd45d/volumes/kubernetes.io~projected/kube-api-access-fhn6w DeviceMajor:0 DeviceMinor:214 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-235 DeviceMajor:0 DeviceMinor:235 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/92a21826-d145-4b9b-9897-aaf21f63a67b/volumes/kubernetes.io~projected/kube-api-access-c2fqp DeviceMajor:0 DeviceMinor:67 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/114fbcb8-1004-410d-98bc-dc60f8fcae3c/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-221 DeviceMajor:0 DeviceMinor:221 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f77f8483d1e74bc955cbbc2ba67ad372126d051afb298b5f24534ac009886dd7/userdata/shm DeviceMajor:0 DeviceMinor:229 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e7dce758fa820d22634e473f8ac42aef98a5118f84adc773ff1426d941333629/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/de9b062be5610c482edd33d85e74f7bf16a925f7219417dbf1e2c6e9a9b03485/userdata/shm DeviceMajor:0 DeviceMinor:93 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-223 DeviceMajor:0 DeviceMinor:223 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/74d238c6a8c7ac178725be85164bcec19e4f1316bab30352419cc161f7577d43/userdata/shm DeviceMajor:0 DeviceMinor:79 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a872df7cb2015d7d1c0dca0187e93c6cd6c58d20446ba442a6d9daf70371177e/userdata/shm DeviceMajor:0 DeviceMinor:215 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d93ec858-ae33-473a-ad4b-18b88664094c/volumes/kubernetes.io~projected/kube-api-access-dhx9h DeviceMajor:0 DeviceMinor:66 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9156ceacadc3ef5c17ff0a82459481ea6c940b128e65c276ddb10d0106554d53/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-204 DeviceMajor:0 DeviceMinor:204 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9963f1102736ee5cf0c733cf42c99fd2f2cec60d380ed11e1d351fed5622cbce/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a517bba6-536f-4dac-ba8b-2e71964ef8c5/volumes/kubernetes.io~projected/kube-api-access-vnjp7 DeviceMajor:0 DeviceMinor:64 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3ebae1f01b86ba4f27b38d27f5f5b48f9d53d221f8cf0665322b56c1e4591869/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-206 DeviceMajor:0 DeviceMinor:206 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9afdc202-0860-42d2-99dc-e1e028b75953/volumes/kubernetes.io~projected/kube-api-access-sz7ff DeviceMajor:0 DeviceMinor:213 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ab0116fc-0400-4f7d-be91-27804b24c8f4/volumes/kubernetes.io~projected/kube-api-access-tgj6t DeviceMajor:0 DeviceMinor:57 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/d93ec858-ae33-473a-ad4b-18b88664094c/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:63 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/36d94e2489c4d23d3321b3cf5a02717ffbd01c4d39715c1e26d42eaf340cdbdc/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-154 DeviceMajor:0 DeviceMinor:154 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b5e430f01d6ae61d520d3977d629b37e08265258a9b1949032de9568b1f0dc51/userdata/shm DeviceMajor:0 DeviceMinor:219 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/11db0a9a-1a94-430c-b395-6ea6bbeba528/volumes/kubernetes.io~projected/kube-api-access-zqfr4 DeviceMajor:0 DeviceMinor:227 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3/volumes/kubernetes.io~projected/kube-api-access-p5cqd DeviceMajor:0 DeviceMinor:228 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3da38b7898a74e5db9cfbea383d4ba1963ac2528558e42bac58423fd58d14ac0/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7cbbb3a060e45b04b442cd03dd28b9df91f2132707a0bcf475d14a91a6ae994b/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f51455e1-dbba-4186-adbf-60f44c9edbe5/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:208 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-231 DeviceMajor:0 DeviceMinor:231 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4a6021d6-6317-473f-97bc-ea023261d90b/volumes/kubernetes.io~projected/kube-api-access-npfv7 DeviceMajor:0 DeviceMinor:65 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f98872b6-9c55-467c-9c70-2831de3fb841/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/114fbcb8-1004-410d-98bc-dc60f8fcae3c/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:55 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/114fbcb8-1004-410d-98bc-dc60f8fcae3c/volumes/kubernetes.io~projected/kube-api-access-pphxn DeviceMajor:0 DeviceMinor:58 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d93ec858-ae33-473a-ad4b-18b88664094c/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:61 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cf3f35db32a715abe6d6fb9ec14c22d8e0bb478facd34ccc963be24ac3d0df58/userdata/shm DeviceMajor:0 DeviceMinor:217 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/90c6593d-b40a-43ae-abc0-ce015e48c289/volumes/kubernetes.io~projected/kube-api-access-8clvh DeviceMajor:0 DeviceMinor:59 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f7894bc9-1a77-42e3-a59b-926729c93dc9/volumes/kubernetes.io~projected/kube-api-access-bdsjl DeviceMajor:0 DeviceMinor:60 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/299ac5c9-51d1-4903-b7f1-2001aa7b2bcf/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:62 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/197507b0c77d583cadd2a8de2b1452d42ccb42e153c5d06b52cde51bb4a195e6/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5db1f514-bc37-424a-89ce-71dbc6a520e9/volumes/kubernetes.io~projected/kube-api-access-j82vx DeviceMajor:0 DeviceMinor:197 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f98872b6-9c55-467c-9c70-2831de3fb841/volumes/kubernetes.io~projected/kube-api-access-vn988 DeviceMajor:0 DeviceMinor:201 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/d7ebad18-eccb-47c9-b259-67c6d6b84d0f/volumes/kubernetes.io~projected/kube-api-access-2hwhq DeviceMajor:0 DeviceMinor:56 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9e8c259aaafc99b31cb10786e94629f2628ff0a257e257b79dd1858155e43d55/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:7cbbb3a060e45b0 MacAddress:c2:2a:8a:59:3e:5c Speed:10000 Mtu:8901} {Name:a872df7cb2015d7 MacAddress:9e:bd:bc:f2:1b:f5 Speed:10000 Mtu:8901} {Name:b5e430f01d6ae61 MacAddress:4a:17:a5:68:36:5e Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:eb:79:f9:57 Speed:0 Mtu:9001} {Name:br-int MacAddress:2a:2a:14:d2:e3:51 Speed:0 Mtu:8901} {Name:cf3f35db32a715a MacAddress:f2:a2:52:e1:d5:9a Speed:10000 Mtu:8901} {Name:ens5 MacAddress:06:ff:eb:79:f9:57 Speed:0 Mtu:9001} {Name:f77f8483d1e74bc MacAddress:0e:04:74:a1:1f:7f Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:0e:50:8a:34:09:2b Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:26:f9:ae:9b:d0:aa Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.542537    4544 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.542627    4544 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.543348    4544 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.543375    4544 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-122.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.543565    4544 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.543579    4544 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.543592    4544 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.543641    4544 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.543752    4544 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.543966    4544 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.544112    4544 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.544154    4544 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.544173    4544 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.544187    4544 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.544199    4544 kubelet.go:405] "Adding apiserver pod source"
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.544210    4544 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.545098    4544 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.545111    4544 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:36:42.546586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.545622    4544 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 18:36:42.547577 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.547563    4544 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 18:36:42.547577 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.547576    4544 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:36:42.547717 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.547695    4544 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 18:36:42.547888 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.547876    4544 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 18:36:42.547917 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.547906    4544 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 18:36:42.548000 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.547992    4544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 18:36:42.548028 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.548005    4544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 18:36:42.548028 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.548013    4544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 18:36:42.548028 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.548018    4544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 18:36:42.548028 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.548024    4544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 18:36:42.548125 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.548030    4544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 18:36:42.548125 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.548035    4544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 18:36:42.548125 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.548041    4544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 18:36:42.548125 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.548047    4544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 18:36:42.548125 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.548059    4544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 18:36:42.548125 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.548072    4544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 18:36:42.548125 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.548082    4544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 18:36:42.548125 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.548104    4544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 18:36:42.548125 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.548110    4544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 18:36:42.551361 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.551350    4544 server.go:1263] "Started kubelet"
Oct 30 18:36:42.551764 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.551425    4544 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 18:36:42.551764 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.551464    4544 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 18:36:42.551764 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.551550    4544 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 18:36:42.551924 ip-10-0-140-122 systemd[1]: Started Kubernetes Kubelet.
Oct 30 18:36:42.552121 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.552009    4544 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 18:36:42.553075 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.552789    4544 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 18:36:42.555981 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.555964    4544 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 18:36:42.556280 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.556254    4544 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 18:36:42.556890 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:42.556870    4544 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-122.us-west-2.compute.internal\" not found"
Oct 30 18:36:42.557028 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.557017    4544 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 18:36:42.557106 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.557097    4544 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 18:36:42.557526 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.557507    4544 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 18:36:42.558196 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.558157    4544 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:31:03 +0000 UTC" deadline="2028-07-13 12:26:30.84926204 +0000 UTC"
Oct 30 18:36:42.558301 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.558287    4544 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14921h49m48.290980205s"
Oct 30 18:36:42.558744 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.558720    4544 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 18:36:42.558744 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.558737    4544 factory.go:55] Registering systemd factory
Oct 30 18:36:42.558744 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.558747    4544 factory.go:223] Registration of the systemd container factory successfully
Oct 30 18:36:42.559243 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.559226    4544 factory.go:153] Registering CRI-O factory
Oct 30 18:36:42.559315 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.559247    4544 factory.go:223] Registration of the crio container factory successfully
Oct 30 18:36:42.559315 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.559265    4544 factory.go:103] Registering Raw factory
Oct 30 18:36:42.559315 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.559303    4544 manager.go:1199] Started watching for new ooms in manager
Oct 30 18:36:42.559845 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.559826    4544 manager.go:319] Starting recovery of all containers
Oct 30 18:36:42.560036 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.559983    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a517bba6-536f-4dac-ba8b-2e71964ef8c5" volumeName="kubernetes.io/projected/a517bba6-536f-4dac-ba8b-2e71964ef8c5-kube-api-access-vnjp7" seLinuxMountContext=""
Oct 30 18:36:42.560096 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560040    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7894bc9-1a77-42e3-a59b-926729c93dc9" volumeName="kubernetes.io/projected/f7894bc9-1a77-42e3-a59b-926729c93dc9-kube-api-access-bdsjl" seLinuxMountContext=""
Oct 30 18:36:42.560096 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560057    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f98872b6-9c55-467c-9c70-2831de3fb841" volumeName="kubernetes.io/secret/f98872b6-9c55-467c-9c70-2831de3fb841-cert" seLinuxMountContext=""
Oct 30 18:36:42.560096 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560078    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11db0a9a-1a94-430c-b395-6ea6bbeba528" volumeName="kubernetes.io/projected/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-api-access-zqfr4" seLinuxMountContext=""
Oct 30 18:36:42.560241 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560096    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5db1f514-bc37-424a-89ce-71dbc6a520e9" volumeName="kubernetes.io/projected/5db1f514-bc37-424a-89ce-71dbc6a520e9-kube-api-access-j82vx" seLinuxMountContext=""
Oct 30 18:36:42.560241 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560117    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="114fbcb8-1004-410d-98bc-dc60f8fcae3c" volumeName="kubernetes.io/projected/114fbcb8-1004-410d-98bc-dc60f8fcae3c-kube-api-access-pphxn" seLinuxMountContext=""
Oct 30 18:36:42.560241 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560132    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11db0a9a-1a94-430c-b395-6ea6bbeba528" volumeName="kubernetes.io/empty-dir/11db0a9a-1a94-430c-b395-6ea6bbeba528-data-volume" seLinuxMountContext=""
Oct 30 18:36:42.560241 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560145    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a517bba6-536f-4dac-ba8b-2e71964ef8c5" volumeName="kubernetes.io/configmap/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-daemon-config" seLinuxMountContext=""
Oct 30 18:36:42.560241 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560167    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab0116fc-0400-4f7d-be91-27804b24c8f4" volumeName="kubernetes.io/configmap/ab0116fc-0400-4f7d-be91-27804b24c8f4-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 18:36:42.560241 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560182    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90c6593d-b40a-43ae-abc0-ce015e48c289" volumeName="kubernetes.io/projected/90c6593d-b40a-43ae-abc0-ce015e48c289-kube-api-access-8clvh" seLinuxMountContext=""
Oct 30 18:36:42.560241 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560201    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab0116fc-0400-4f7d-be91-27804b24c8f4" volumeName="kubernetes.io/projected/ab0116fc-0400-4f7d-be91-27804b24c8f4-kube-api-access-tgj6t" seLinuxMountContext=""
Oct 30 18:36:42.560241 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560216    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a6021d6-6317-473f-97bc-ea023261d90b" volumeName="kubernetes.io/configmap/4a6021d6-6317-473f-97bc-ea023261d90b-serviceca" seLinuxMountContext=""
Oct 30 18:36:42.560241 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560234    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3" volumeName="kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-service-ca-bundle" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560250    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3" volumeName="kubernetes.io/empty-dir/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-archives-path" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560273    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab0116fc-0400-4f7d-be91-27804b24c8f4" volumeName="kubernetes.io/configmap/ab0116fc-0400-4f7d-be91-27804b24c8f4-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560287    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d93ec858-ae33-473a-ad4b-18b88664094c" volumeName="kubernetes.io/empty-dir/d93ec858-ae33-473a-ad4b-18b88664094c-etc-tuned" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560300    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a6021d6-6317-473f-97bc-ea023261d90b" volumeName="kubernetes.io/projected/4a6021d6-6317-473f-97bc-ea023261d90b-kube-api-access-npfv7" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560318    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="92a21826-d145-4b9b-9897-aaf21f63a67b" volumeName="kubernetes.io/projected/92a21826-d145-4b9b-9897-aaf21f63a67b-kube-api-access-c2fqp" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560333    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9afdc202-0860-42d2-99dc-e1e028b75953" volumeName="kubernetes.io/projected/9afdc202-0860-42d2-99dc-e1e028b75953-kube-api-access-sz7ff" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560351    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3" volumeName="kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560368    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ab0116fc-0400-4f7d-be91-27804b24c8f4" volumeName="kubernetes.io/configmap/ab0116fc-0400-4f7d-be91-27804b24c8f4-cni-binary-copy" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560381    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d93ec858-ae33-473a-ad4b-18b88664094c" volumeName="kubernetes.io/empty-dir/d93ec858-ae33-473a-ad4b-18b88664094c-tmp" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560398    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f3e9afad-06d1-42eb-b463-5dc5932cd45d" volumeName="kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560415    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f7894bc9-1a77-42e3-a59b-926729c93dc9" volumeName="kubernetes.io/configmap/f7894bc9-1a77-42e3-a59b-926729c93dc9-iptables-alerter-script" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560436    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="114fbcb8-1004-410d-98bc-dc60f8fcae3c" volumeName="kubernetes.io/secret/114fbcb8-1004-410d-98bc-dc60f8fcae3c-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560451    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="299ac5c9-51d1-4903-b7f1-2001aa7b2bcf" volumeName="kubernetes.io/secret/299ac5c9-51d1-4903-b7f1-2001aa7b2bcf-agent-certs" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560500    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5db1f514-bc37-424a-89ce-71dbc6a520e9" volumeName="kubernetes.io/configmap/5db1f514-bc37-424a-89ce-71dbc6a520e9-config-volume" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560519    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5db1f514-bc37-424a-89ce-71dbc6a520e9" volumeName="kubernetes.io/empty-dir/5db1f514-bc37-424a-89ce-71dbc6a520e9-tmp-dir" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560546    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a517bba6-536f-4dac-ba8b-2e71964ef8c5" volumeName="kubernetes.io/configmap/a517bba6-536f-4dac-ba8b-2e71964ef8c5-cni-binary-copy" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560560    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3" volumeName="kubernetes.io/projected/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-kube-api-access-p5cqd" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560579    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d7ebad18-eccb-47c9-b259-67c6d6b84d0f" volumeName="kubernetes.io/projected/d7ebad18-eccb-47c9-b259-67c6d6b84d0f-kube-api-access-2hwhq" seLinuxMountContext=""
Oct 30 18:36:42.560668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560593    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d93ec858-ae33-473a-ad4b-18b88664094c" volumeName="kubernetes.io/projected/d93ec858-ae33-473a-ad4b-18b88664094c-kube-api-access-dhx9h" seLinuxMountContext=""
Oct 30 18:36:42.561539 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560737    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="114fbcb8-1004-410d-98bc-dc60f8fcae3c" volumeName="kubernetes.io/configmap/114fbcb8-1004-410d-98bc-dc60f8fcae3c-env-overrides" seLinuxMountContext=""
Oct 30 18:36:42.561539 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560756    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="114fbcb8-1004-410d-98bc-dc60f8fcae3c" volumeName="kubernetes.io/configmap/114fbcb8-1004-410d-98bc-dc60f8fcae3c-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 18:36:42.561539 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560773    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="114fbcb8-1004-410d-98bc-dc60f8fcae3c" volumeName="kubernetes.io/configmap/114fbcb8-1004-410d-98bc-dc60f8fcae3c-ovnkube-config" seLinuxMountContext=""
Oct 30 18:36:42.561539 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560793    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="299ac5c9-51d1-4903-b7f1-2001aa7b2bcf" volumeName="kubernetes.io/configmap/299ac5c9-51d1-4903-b7f1-2001aa7b2bcf-konnectivity-ca" seLinuxMountContext=""
Oct 30 18:36:42.561539 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560806    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d7ebad18-eccb-47c9-b259-67c6d6b84d0f" volumeName="kubernetes.io/empty-dir/d7ebad18-eccb-47c9-b259-67c6d6b84d0f-tmp-dir" seLinuxMountContext=""
Oct 30 18:36:42.561539 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560818    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f51455e1-dbba-4186-adbf-60f44c9edbe5" volumeName="kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret" seLinuxMountContext=""
Oct 30 18:36:42.561539 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560835    4544 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f98872b6-9c55-467c-9c70-2831de3fb841" volumeName="kubernetes.io/projected/f98872b6-9c55-467c-9c70-2831de3fb841-kube-api-access-vn988" seLinuxMountContext=""
Oct 30 18:36:42.561539 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560851    4544 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 18:36:42.561539 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.560858    4544 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 18:36:42.580195 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.580173    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 18:36:42.580302 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.580234    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 18:36:42.591830 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.591796    4544 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 18:36:42.594724 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.594703    4544 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 18:36:42.594724 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.594724    4544 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 18:36:42.594873 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.594743    4544 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 18:36:42.594873 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:42.594781    4544 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 18:36:42.606380 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.606350    4544 generic.go:356] "Generic (PLEG): container finished" podID="ab0116fc-0400-4f7d-be91-27804b24c8f4" containerID="91a13660483839a894e8962d0616b503a04e946757cc61ca39ac4eb81a703f81" exitCode=0
Oct 30 18:36:42.606380 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.606377    4544 generic.go:356] "Generic (PLEG): container finished" podID="ab0116fc-0400-4f7d-be91-27804b24c8f4" containerID="34b3448b05c4a5dae8f7d6a1e438d2a3fba26544d449c8dd757f6df3b679f907" exitCode=0
Oct 30 18:36:42.606524 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.606389    4544 generic.go:356] "Generic (PLEG): container finished" podID="ab0116fc-0400-4f7d-be91-27804b24c8f4" containerID="5d632c16162f5c46b761a49a0c3d7a0ba06b6083fd5f5cbdaa25fbd39f32e93d" exitCode=0
Oct 30 18:36:42.606524 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.606398    4544 generic.go:356] "Generic (PLEG): container finished" podID="ab0116fc-0400-4f7d-be91-27804b24c8f4" containerID="f43c14c280c7f015f06d54695f2ee9572fa9b0d833e56267b009808ffad8d6d8" exitCode=0
Oct 30 18:36:42.606524 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.606406    4544 generic.go:356] "Generic (PLEG): container finished" podID="ab0116fc-0400-4f7d-be91-27804b24c8f4" containerID="b21965d88dfebf7911fb927d0aed491c9a10f2bdf8478fee636617e96fae8c93" exitCode=0
Oct 30 18:36:42.606524 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.606413    4544 generic.go:356] "Generic (PLEG): container finished" podID="ab0116fc-0400-4f7d-be91-27804b24c8f4" containerID="dd4ce462f2d806b8c19db8ca9309f64882797f045063c171cfbd650da3a66ba0" exitCode=0
Oct 30 18:36:42.611387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.611362    4544 generic.go:356] "Generic (PLEG): container finished" podID="7d48fbbc6436568b80ec3382fdeed930" containerID="05994d6179cee66ea6ec258129e44cb3cf95e525fa4a2f6da3778f6c700730ef" exitCode=0
Oct 30 18:36:42.629338 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.629310    4544 manager.go:324] Recovery completed
Oct 30 18:36:42.629770 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.629753    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 18:36:42.630072 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.630050    4544 generic.go:356] "Generic (PLEG): container finished" podID="114fbcb8-1004-410d-98bc-dc60f8fcae3c" containerID="03b05f32b6c8e5315b71345369389005b85e9ec8d87fbec3189b0cb8fdc488e3" exitCode=1
Oct 30 18:36:42.642135 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.642115    4544 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:42.643315 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.643300    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:42.643385 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.643328    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:42.643385 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.643338    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:36:42.644627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.644593    4544 cpu_manager.go:226] "Starting" policy="none"
Oct 30 18:36:42.644686 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.644631    4544 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 18:36:42.644686 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.644650    4544 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 18:36:42.644739 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.644618    4544 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:42.644794 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.644778    4544 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 18:36:42.644831 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.644797    4544 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 18:36:42.644831 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.644820    4544 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 18:36:42.644831 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.644828    4544 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 18:36:42.644908 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.644836    4544 policy_none.go:50] "Start"
Oct 30 18:36:42.644908 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.644848    4544 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 18:36:42.644908 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.644861    4544 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 18:36:42.644997 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.644988    4544 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 18:36:42.645026 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.644998    4544 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 18:36:42.645026 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.645004    4544 policy_none.go:44] "Start"
Oct 30 18:36:42.645678 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.645664    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:42.645719 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.645696    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:42.645719 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.645711    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:36:42.649330 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.649279    4544 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 18:36:42.652488 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.652473    4544 manager.go:341] "Starting Device Plugin manager"
Oct 30 18:36:42.652569 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:42.652500    4544 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 18:36:42.652569 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.652508    4544 server.go:87] "Starting device plugin registration server"
Oct 30 18:36:42.652815 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.652802    4544 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 18:36:42.652845 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.652817    4544 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 18:36:42.652949 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.652925    4544 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 18:36:42.653036 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.653011    4544 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 18:36:42.653082 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.653041    4544 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 18:36:42.653082 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.653048    4544 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 18:36:42.653179 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.653106    4544 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:36:42.653031058Z","UUID":"12d87020-90f8-43ab-897f-2cfc8460c0af","Handler":null,"Name":"","Endpoint":""}
Oct 30 18:36:42.654330 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:42.654310    4544 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 18:36:42.654417 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:42.654351    4544 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-122.us-west-2.compute.internal\" not found"
Oct 30 18:36:42.655055 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.654811    4544 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 18:36:42.655055 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.654836    4544 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 18:36:42.695306 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.694847    4544 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal"]
Oct 30 18:36:42.695419 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.695408    4544 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:42.696545 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.696530    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:42.696634 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.696556    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:42.696634 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.696567    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:36:42.696815 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.696805    4544 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:42.696960 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.696935    4544 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:42.698029 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.698010    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:42.698115 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.698045    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:42.698115 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.698061    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:36:42.698115 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.698013    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:42.698115 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.698101    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:42.698115 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.698115    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:36:42.698283 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:42.698268    4544 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-122.us-west-2.compute.internal\" not found" node="ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:42.698371 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.698347    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal" event={"ID":"7d48fbbc6436568b80ec3382fdeed930","Type":"ContainerStarted","Data":"71bef265f12a68f9ea81389338f12da88bc07185618bfbdd72c4dda0be9eeea7"}
Oct 30 18:36:42.698401 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.698384    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal" event={"ID":"7d48fbbc6436568b80ec3382fdeed930","Type":"ContainerDied","Data":"05994d6179cee66ea6ec258129e44cb3cf95e525fa4a2f6da3778f6c700730ef"}
Oct 30 18:36:42.698429 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.698400    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal" event={"ID":"7d48fbbc6436568b80ec3382fdeed930","Type":"ContainerStarted","Data":"3da38b7898a74e5db9cfbea383d4ba1963ac2528558e42bac58423fd58d14ac0"}
Oct 30 18:36:42.698429 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.698411    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal" event={"ID":"7018e001fd99c1f4ae76683b64500513","Type":"ContainerStarted","Data":"edceb83d17fc64693db7bf63791d26332bcec6c2cc54e58b0610edc21e42f588"}
Oct 30 18:36:42.698429 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.698423    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal" event={"ID":"7018e001fd99c1f4ae76683b64500513","Type":"ContainerStarted","Data":"9963f1102736ee5cf0c733cf42c99fd2f2cec60d380ed11e1d351fed5622cbce"}
Oct 30 18:36:42.698506 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.698430    4544 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:42.699393 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.699375    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:42.699458 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.699402    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:42.699458 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.699411    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:36:42.699541 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:42.699529    4544 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-122.us-west-2.compute.internal\" not found" node="ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:42.742560 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.742535    4544 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:42.742652 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.742588    4544 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 18:36:42.743913 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.743897    4544 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:42.753568 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.753547    4544 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:42.759113 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.758935    4544 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:42.762835 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.762820    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:42.762900 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.762848    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:42.762900 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.762859    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:36:42.762900 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.762889    4544 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:42.788667 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.788649    4544 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:42.790583 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.790562    4544 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:42.831399 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.831379    4544 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:36:42.831470 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:42.831417    4544 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:42.831470 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.831429    4544 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:42.842920 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.842898    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7d48fbbc6436568b80ec3382fdeed930-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal\" (UID: \"7d48fbbc6436568b80ec3382fdeed930\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:42.842998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.842923    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d48fbbc6436568b80ec3382fdeed930-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal\" (UID: \"7d48fbbc6436568b80ec3382fdeed930\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:42.842998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.842942    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7018e001fd99c1f4ae76683b64500513-config\") pod \"kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal\" (UID: \"7018e001fd99c1f4ae76683b64500513\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:42.880912 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.880892    4544 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:42.880980 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.880948    4544 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:42.881412 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.881398    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeNotReady"
Oct 30 18:36:42.881477 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.881424    4544 setters.go:546] "Node became not ready" node="ip-10-0-140-122.us-west-2.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-10-30T18:36:42Z","lastTransitionTime":"2026-10-30T18:36:42Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Oct 30 18:36:42.886819 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.886802    4544 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:36:42.886864 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:42.886837    4544 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:42.896324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.896303    4544 nodeinfomanager.go:471] existing CSINode "ip-10-0-140-122.us-west-2.compute.internal" is owned by different node (oldNodeID="e88224a7-1700-4f3c-8121-c9c3e45cd812", newNodeID=""), cleaning up...
Oct 30 18:36:42.943620 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.943550    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7d48fbbc6436568b80ec3382fdeed930-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal\" (UID: \"7d48fbbc6436568b80ec3382fdeed930\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:42.943620 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.943580    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d48fbbc6436568b80ec3382fdeed930-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal\" (UID: \"7d48fbbc6436568b80ec3382fdeed930\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:42.943728 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.943623    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7018e001fd99c1f4ae76683b64500513-config\") pod \"kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal\" (UID: \"7018e001fd99c1f4ae76683b64500513\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:42.943728 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.943660    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7d48fbbc6436568b80ec3382fdeed930-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal\" (UID: \"7d48fbbc6436568b80ec3382fdeed930\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:42.943728 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.943672    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d48fbbc6436568b80ec3382fdeed930-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal\" (UID: \"7d48fbbc6436568b80ec3382fdeed930\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:42.943728 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:42.943665    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7018e001fd99c1f4ae76683b64500513-config\") pod \"kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal\" (UID: \"7018e001fd99c1f4ae76683b64500513\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:43.544636 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.544590    4544 apiserver.go:52] "Watching apiserver"
Oct 30 18:36:43.634228 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.634199    4544 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:43.634363 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.634326    4544 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:43.641524 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.641504    4544 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 18:36:43.641982 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.641960    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal","kube-system/global-pull-secret-syncer-b4fj5","openshift-image-registry/node-ca-qkvdd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal","openshift-network-diagnostics/network-check-target-qk8z2","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz","openshift-insights/insights-runtime-extractor-6zrww","openshift-dns/dns-default-zmj6x","openshift-kube-storage-version-migrator/migrator-66df5d496b-nr9vn","openshift-multus/multus-additional-cni-plugins-cxhd5","openshift-network-operator/iptables-alerter-nssbw","openshift-cluster-node-tuning-operator/tuned-78mjc","openshift-insights/periodic-gathering-pntg4-8mc75","openshift-ingress-canary/ingress-canary-hwxx2","openshift-multus/network-metrics-daemon-7wdcf","kube-system/konnectivity-agent-r52tw","openshift-dns/node-resolver-nmmwh","openshift-multus/multus-54jpr","openshift-ovn-kubernetes/ovnkube-node-nxnnz"]
Oct 30 18:36:43.647275 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.647250    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:43.648660 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.648641    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:43.650262 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.650241    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:43.658450 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.658436    4544 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 18:36:43.692066 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.692045    4544 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:36:43.692139 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:43.692104    4544 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:43.692223 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.692209    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-5mz5g\""
Oct 30 18:36:43.692854 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.692837    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 18:36:43.692943 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.692914    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 18:36:43.692943 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.692923    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 18:36:43.692943 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.692932    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 18:36:43.692943 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.692936    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 18:36:43.693376 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.693359    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m6qzp\""
Oct 30 18:36:43.693799 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.693786    4544 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:36:43.693854 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:43.693840    4544 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-122.us-west-2.compute.internal"
Oct 30 18:36:43.693908 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.693878    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 18:36:43.693908 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.693897    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 18:36:43.693982 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.693911    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 18:36:43.694908 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.694893    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 18:36:43.695335 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.695321    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-h4btl\""
Oct 30 18:36:43.696077 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.696055    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 18:36:43.696190 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.696126    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-5slhp\""
Oct 30 18:36:43.696190 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.696164    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-9ph7g\""
Oct 30 18:36:43.696190 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.696166    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 18:36:43.696339 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.696225    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-7kzh2\""
Oct 30 18:36:43.696339 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.696130    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j6hdp\""
Oct 30 18:36:43.696524 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.696506    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tlfn7\""
Oct 30 18:36:43.696591 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.696575    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 18:36:43.696671 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.696657    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cxkbs\""
Oct 30 18:36:43.696710 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.696697    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vpn2j\""
Oct 30 18:36:43.696753 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.696743    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 18:36:43.729720 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.729696    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 18:36:43.730295 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.730274    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 18:36:43.730295 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.730288    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 18:36:43.730449 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.730322    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vtx29\""
Oct 30 18:36:43.730449 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.730346    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-xg84v\""
Oct 30 18:36:43.730539 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.730487    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-zmfwz\""
Oct 30 18:36:43.730626 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.730588    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dnpxx\""
Oct 30 18:36:43.730819 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.730803    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-mbxkm\""
Oct 30 18:36:43.730908 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.730895    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-swngw\""
Oct 30 18:36:43.730944 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.730939    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vd777\""
Oct 30 18:36:43.747735 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.747716    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5db1f514-bc37-424a-89ce-71dbc6a520e9-config-volume\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:43.747814 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.747744    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-socket-dir-parent\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.747814 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.747763    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-var-lib-kubelet\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.747814 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.747780    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d93ec858-ae33-473a-ad4b-18b88664094c-tmp\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.747942 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.747852    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-service-ca-bundle\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:43.747942 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.747882    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-etc-selinux\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.747942 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.747901    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqfr4\" (UniqueName: \"kubernetes.io/projected/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-api-access-zqfr4\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:43.747942 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.747922    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4a6021d6-6317-473f-97bc-ea023261d90b-serviceca\") pod \"node-ca-qkvdd\" (UID: \"4a6021d6-6317-473f-97bc-ea023261d90b\") " pod="openshift-image-registry/node-ca-qkvdd"
Oct 30 18:36:43.748076 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.747981    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7894bc9-1a77-42e3-a59b-926729c93dc9-iptables-alerter-script\") pod \"iptables-alerter-nssbw\" (UID: \"f7894bc9-1a77-42e3-a59b-926729c93dc9\") " pod="openshift-network-operator/iptables-alerter-nssbw"
Oct 30 18:36:43.748076 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748004    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d93ec858-ae33-473a-ad4b-18b88664094c-tmp\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.748076 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748013    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-trusted-ca-bundle\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:43.748076 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748055    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab0116fc-0400-4f7d-be91-27804b24c8f4-cnibin\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:43.748219 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748094    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab0116fc-0400-4f7d-be91-27804b24c8f4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:43.748219 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748127    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2fqp\" (UniqueName: \"kubernetes.io/projected/92a21826-d145-4b9b-9897-aaf21f63a67b-kube-api-access-c2fqp\") pod \"network-metrics-daemon-7wdcf\" (UID: \"92a21826-d145-4b9b-9897-aaf21f63a67b\") " pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:43.748219 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748158    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdsjl\" (UniqueName: \"kubernetes.io/projected/f7894bc9-1a77-42e3-a59b-926729c93dc9-kube-api-access-bdsjl\") pod \"iptables-alerter-nssbw\" (UID: \"f7894bc9-1a77-42e3-a59b-926729c93dc9\") " pod="openshift-network-operator/iptables-alerter-nssbw"
Oct 30 18:36:43.748329 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748229    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-system-cni-dir\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.748329 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748304    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz7ff\" (UniqueName: \"kubernetes.io/projected/9afdc202-0860-42d2-99dc-e1e028b75953-kube-api-access-sz7ff\") pod \"migrator-66df5d496b-nr9vn\" (UID: \"9afdc202-0860-42d2-99dc-e1e028b75953\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nr9vn"
Oct 30 18:36:43.748399 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748371    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-run-ovn\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.748440 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748404    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/114fbcb8-1004-410d-98bc-dc60f8fcae3c-env-overrides\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.748440 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748411    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4a6021d6-6317-473f-97bc-ea023261d90b-serviceca\") pod \"node-ca-qkvdd\" (UID: \"4a6021d6-6317-473f-97bc-ea023261d90b\") " pod="openshift-image-registry/node-ca-qkvdd"
Oct 30 18:36:43.748440 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748431    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/114fbcb8-1004-410d-98bc-dc60f8fcae3c-ovn-node-metrics-cert\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.748586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748475    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d7ebad18-eccb-47c9-b259-67c6d6b84d0f-hosts-file\") pod \"node-resolver-nmmwh\" (UID: \"d7ebad18-eccb-47c9-b259-67c6d6b84d0f\") " pod="openshift-dns/node-resolver-nmmwh"
Oct 30 18:36:43.748586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748540    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hwhq\" (UniqueName: \"kubernetes.io/projected/d7ebad18-eccb-47c9-b259-67c6d6b84d0f-kube-api-access-2hwhq\") pod \"node-resolver-nmmwh\" (UID: \"d7ebad18-eccb-47c9-b259-67c6d6b84d0f\") " pod="openshift-dns/node-resolver-nmmwh"
Oct 30 18:36:43.748586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748566    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-sysctl-conf\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.748731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748691    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/114fbcb8-1004-410d-98bc-dc60f8fcae3c-ovn-node-metrics-cert\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.748781 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748758    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-archives-path\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:43.748781 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748596    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-archives-path\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:43.748844 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748808    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgj6t\" (UniqueName: \"kubernetes.io/projected/ab0116fc-0400-4f7d-be91-27804b24c8f4-kube-api-access-tgj6t\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:43.748879 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748850    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-sys-fs\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.748914 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748874    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-etc-kubernetes\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.748960 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748924    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhn6w\" (UniqueName: \"kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w\") pod \"network-check-target-qk8z2\" (UID: \"f3e9afad-06d1-42eb-b463-5dc5932cd45d\") " pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:43.749012 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.748949    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-kubernetes\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.749012 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749004    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8clvh\" (UniqueName: \"kubernetes.io/projected/90c6593d-b40a-43ae-abc0-ce015e48c289-kube-api-access-8clvh\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.749099 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749023    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f51455e1-dbba-4186-adbf-60f44c9edbe5-kubelet-config\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:43.749099 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749057    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:43.749201 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749182    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs\") pod \"network-metrics-daemon-7wdcf\" (UID: \"92a21826-d145-4b9b-9897-aaf21f63a67b\") " pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:43.749279 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749230    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5db1f514-bc37-424a-89ce-71dbc6a520e9-tmp-dir\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:43.749279 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749261    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn988\" (UniqueName: \"kubernetes.io/projected/f98872b6-9c55-467c-9c70-2831de3fb841-kube-api-access-vn988\") pod \"ingress-canary-hwxx2\" (UID: \"f98872b6-9c55-467c-9c70-2831de3fb841\") " pod="openshift-ingress-canary/ingress-canary-hwxx2"
Oct 30 18:36:43.749343 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749284    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-hostroot\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.749343 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749305    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnjp7\" (UniqueName: \"kubernetes.io/projected/a517bba6-536f-4dac-ba8b-2e71964ef8c5-kube-api-access-vnjp7\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.749343 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749327    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-var-lib-openvswitch\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.749462 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749370    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5db1f514-bc37-424a-89ce-71dbc6a520e9-tmp-dir\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:43.749462 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749394    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-log-socket\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.749462 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749430    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f51455e1-dbba-4186-adbf-60f44c9edbe5-original-pull-secret\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:43.749462 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749449    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-cni-netd\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.749638 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749470    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/114fbcb8-1004-410d-98bc-dc60f8fcae3c-ovnkube-config\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.749638 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749484    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7894bc9-1a77-42e3-a59b-926729c93dc9-host-slash\") pod \"iptables-alerter-nssbw\" (UID: \"f7894bc9-1a77-42e3-a59b-926729c93dc9\") " pod="openshift-network-operator/iptables-alerter-nssbw"
Oct 30 18:36:43.749638 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749506    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-kubelet\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.749638 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749532    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-run-netns\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.749638 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749569    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-cni-bin\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.749638 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749632    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-systemd\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.749905 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749681    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhx9h\" (UniqueName: \"kubernetes.io/projected/d93ec858-ae33-473a-ad4b-18b88664094c-kube-api-access-dhx9h\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.749905 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749743    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-slash\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.749905 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749858    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-run-ovn-kubernetes\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.750041 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749901    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-lib-modules\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.750041 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.749942    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-host\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.750041 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750008    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-var-lib-cni-multus\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.750180 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750041    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-run-multus-certs\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.750180 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750073    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/299ac5c9-51d1-4903-b7f1-2001aa7b2bcf-agent-certs\") pod \"konnectivity-agent-r52tw\" (UID: \"299ac5c9-51d1-4903-b7f1-2001aa7b2bcf\") " pod="kube-system/konnectivity-agent-r52tw"
Oct 30 18:36:43.750180 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750126    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-sys\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.750180 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750147    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:43.750180 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750161    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-cnibin\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.750397 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750207    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-run-netns\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.750397 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750239    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-var-lib-cni-bin\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.750397 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750260    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-var-lib-kubelet\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.750397 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750299    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-systemd-units\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.750397 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750323    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-run-systemd\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.750397 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750345    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-run-openvswitch\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.750397 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750360    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab0116fc-0400-4f7d-be91-27804b24c8f4-system-cni-dir\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:43.750690 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750431    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-cni-dir\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.750690 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750456    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/299ac5c9-51d1-4903-b7f1-2001aa7b2bcf-agent-certs\") pod \"konnectivity-agent-r52tw\" (UID: \"299ac5c9-51d1-4903-b7f1-2001aa7b2bcf\") " pod="kube-system/konnectivity-agent-r52tw"
Oct 30 18:36:43.750690 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750480    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-node-log\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.750690 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750517    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7ebad18-eccb-47c9-b259-67c6d6b84d0f-tmp-dir\") pod \"node-resolver-nmmwh\" (UID: \"d7ebad18-eccb-47c9-b259-67c6d6b84d0f\") " pod="openshift-dns/node-resolver-nmmwh"
Oct 30 18:36:43.750690 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750552    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-sysctl-d\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.750690 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750581    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab0116fc-0400-4f7d-be91-27804b24c8f4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:43.750690 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750626    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/11db0a9a-1a94-430c-b395-6ea6bbeba528-data-volume\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:43.750690 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750644    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7ebad18-eccb-47c9-b259-67c6d6b84d0f-tmp-dir\") pod \"node-resolver-nmmwh\" (UID: \"d7ebad18-eccb-47c9-b259-67c6d6b84d0f\") " pod="openshift-dns/node-resolver-nmmwh"
Oct 30 18:36:43.750690 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750658    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f98872b6-9c55-467c-9c70-2831de3fb841-cert\") pod \"ingress-canary-hwxx2\" (UID: \"f98872b6-9c55-467c-9c70-2831de3fb841\") " pod="openshift-ingress-canary/ingress-canary-hwxx2"
Oct 30 18:36:43.751090 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750719    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/11db0a9a-1a94-430c-b395-6ea6bbeba528-data-volume\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:43.751090 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750735    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a517bba6-536f-4dac-ba8b-2e71964ef8c5-cni-binary-copy\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.751090 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750812    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/114fbcb8-1004-410d-98bc-dc60f8fcae3c-ovnkube-script-lib\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.751090 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750894    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f98872b6-9c55-467c-9c70-2831de3fb841-cert\") pod \"ingress-canary-hwxx2\" (UID: \"f98872b6-9c55-467c-9c70-2831de3fb841\") " pod="openshift-ingress-canary/ingress-canary-hwxx2"
Oct 30 18:36:43.751090 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750904    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pphxn\" (UniqueName: \"kubernetes.io/projected/114fbcb8-1004-410d-98bc-dc60f8fcae3c-kube-api-access-pphxn\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.751090 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750922    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d93ec858-ae33-473a-ad4b-18b88664094c-etc-tuned\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.751090 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750938    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/11db0a9a-1a94-430c-b395-6ea6bbeba528-crio-socket\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:43.751090 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750953    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:43.751090 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750968    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a6021d6-6317-473f-97bc-ea023261d90b-host\") pod \"node-ca-qkvdd\" (UID: \"4a6021d6-6317-473f-97bc-ea023261d90b\") " pod="openshift-image-registry/node-ca-qkvdd"
Oct 30 18:36:43.751090 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.750983    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-registration-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.751090 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751025    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-modprobe-d\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.751090 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751034    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d93ec858-ae33-473a-ad4b-18b88664094c-etc-tuned\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.751090 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751077    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a517bba6-536f-4dac-ba8b-2e71964ef8c5-cni-binary-copy\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.751573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751118    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-run\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.751573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751168    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab0116fc-0400-4f7d-be91-27804b24c8f4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:43.751573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751188    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.751573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751204    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-device-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.751573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751221    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:43.751573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751288    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npfv7\" (UniqueName: \"kubernetes.io/projected/4a6021d6-6317-473f-97bc-ea023261d90b-kube-api-access-npfv7\") pod \"node-ca-qkvdd\" (UID: \"4a6021d6-6317-473f-97bc-ea023261d90b\") " pod="openshift-image-registry/node-ca-qkvdd"
Oct 30 18:36:43.751573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751305    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/299ac5c9-51d1-4903-b7f1-2001aa7b2bcf-konnectivity-ca\") pod \"konnectivity-agent-r52tw\" (UID: \"299ac5c9-51d1-4903-b7f1-2001aa7b2bcf\") " pod="kube-system/konnectivity-agent-r52tw"
Oct 30 18:36:43.751573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751321    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.751573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751337    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5cqd\" (UniqueName: \"kubernetes.io/projected/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-kube-api-access-p5cqd\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:43.751573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751362    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab0116fc-0400-4f7d-be91-27804b24c8f4-cni-binary-copy\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:43.751573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751390    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-socket-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.751573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751408    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f51455e1-dbba-4186-adbf-60f44c9edbe5-dbus\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:43.751573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751428    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j82vx\" (UniqueName: \"kubernetes.io/projected/5db1f514-bc37-424a-89ce-71dbc6a520e9-kube-api-access-j82vx\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:43.751573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751449    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-run-k8s-cni-cncf-io\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.751573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751465    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-conf-dir\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.751573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751498    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-daemon-config\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.751573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751514    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-etc-openvswitch\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.751573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751539    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-sysconfig\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.752180 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751645    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab0116fc-0400-4f7d-be91-27804b24c8f4-cni-binary-copy\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:43.752180 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.751645    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab0116fc-0400-4f7d-be91-27804b24c8f4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:43.752180 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.752058    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-daemon-config\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.793491 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.793459    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 18:36:43.793665 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.793576    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 18:36:43.793838 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.793804    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 18:36:43.793947 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.793927    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 18:36:43.793996 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.793811    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:43.794209 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.794137    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 18:36:43.794209 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.794195    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 18:36:43.794463 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.794447    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 18:36:43.794519 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.794476    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 18:36:43.794566 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.794549    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 18:36:43.794635 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.794590    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 18:36:43.795223 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.795081    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 18:36:43.795223 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.795157    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 18:36:43.795405 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.795376    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 18:36:43.795658 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.795640    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:43.795856 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.795835    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 18:36:43.795988 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.795889    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 18:36:43.795988 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.795935    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:43.795988 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.795889    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 18:36:43.796139 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.796014    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 18:36:43.796139 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.796128    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 18:36:43.796306 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.796181    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 18:36:43.796306 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.796197    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 18:36:43.796410 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.796308    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 18:36:43.796410 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.796375    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 18:36:43.796410 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.796374    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 18:36:43.796410 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.796406    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 18:36:43.797506 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.797358    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 18:36:43.798670 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.798641    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5db1f514-bc37-424a-89ce-71dbc6a520e9-config-volume\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:43.798756 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.798654    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f7894bc9-1a77-42e3-a59b-926729c93dc9-iptables-alerter-script\") pod \"iptables-alerter-nssbw\" (UID: \"f7894bc9-1a77-42e3-a59b-926729c93dc9\") " pod="openshift-network-operator/iptables-alerter-nssbw"
Oct 30 18:36:43.798756 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.798701    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab0116fc-0400-4f7d-be91-27804b24c8f4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:43.798852 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.798836    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/114fbcb8-1004-410d-98bc-dc60f8fcae3c-env-overrides\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.798893 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.798872    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-service-ca-bundle\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:43.799181 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.799166    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-trusted-ca-bundle\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:43.799762 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.799745    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/114fbcb8-1004-410d-98bc-dc60f8fcae3c-ovnkube-config\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.800411 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.800391    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8clvh\" (UniqueName: \"kubernetes.io/projected/90c6593d-b40a-43ae-abc0-ce015e48c289-kube-api-access-8clvh\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.800546 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.800459    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqfr4\" (UniqueName: \"kubernetes.io/projected/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-api-access-zqfr4\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:43.800813 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.800794    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdsjl\" (UniqueName: \"kubernetes.io/projected/f7894bc9-1a77-42e3-a59b-926729c93dc9-kube-api-access-bdsjl\") pod \"iptables-alerter-nssbw\" (UID: \"f7894bc9-1a77-42e3-a59b-926729c93dc9\") " pod="openshift-network-operator/iptables-alerter-nssbw"
Oct 30 18:36:43.800866 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.800849    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2fqp\" (UniqueName: \"kubernetes.io/projected/92a21826-d145-4b9b-9897-aaf21f63a67b-kube-api-access-c2fqp\") pod \"network-metrics-daemon-7wdcf\" (UID: \"92a21826-d145-4b9b-9897-aaf21f63a67b\") " pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:43.800897 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.800878    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz7ff\" (UniqueName: \"kubernetes.io/projected/9afdc202-0860-42d2-99dc-e1e028b75953-kube-api-access-sz7ff\") pod \"migrator-66df5d496b-nr9vn\" (UID: \"9afdc202-0860-42d2-99dc-e1e028b75953\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nr9vn"
Oct 30 18:36:43.800955 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.800939    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgj6t\" (UniqueName: \"kubernetes.io/projected/ab0116fc-0400-4f7d-be91-27804b24c8f4-kube-api-access-tgj6t\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:43.800994 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.800951    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hwhq\" (UniqueName: \"kubernetes.io/projected/d7ebad18-eccb-47c9-b259-67c6d6b84d0f-kube-api-access-2hwhq\") pod \"node-resolver-nmmwh\" (UID: \"d7ebad18-eccb-47c9-b259-67c6d6b84d0f\") " pod="openshift-dns/node-resolver-nmmwh"
Oct 30 18:36:43.801105 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.801090    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/114fbcb8-1004-410d-98bc-dc60f8fcae3c-ovnkube-script-lib\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.801819 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.801803    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnjp7\" (UniqueName: \"kubernetes.io/projected/a517bba6-536f-4dac-ba8b-2e71964ef8c5-kube-api-access-vnjp7\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.801918 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.801903    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhn6w\" (UniqueName: \"kubernetes.io/projected/f3e9afad-06d1-42eb-b463-5dc5932cd45d-kube-api-access-fhn6w\") pod \"network-check-target-qk8z2\" (UID: \"f3e9afad-06d1-42eb-b463-5dc5932cd45d\") " pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:43.801969 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.801927    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/299ac5c9-51d1-4903-b7f1-2001aa7b2bcf-konnectivity-ca\") pod \"konnectivity-agent-r52tw\" (UID: \"299ac5c9-51d1-4903-b7f1-2001aa7b2bcf\") " pod="kube-system/konnectivity-agent-r52tw"
Oct 30 18:36:43.802364 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.802342    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhx9h\" (UniqueName: \"kubernetes.io/projected/d93ec858-ae33-473a-ad4b-18b88664094c-kube-api-access-dhx9h\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.802693 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.802674    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j82vx\" (UniqueName: \"kubernetes.io/projected/5db1f514-bc37-424a-89ce-71dbc6a520e9-kube-api-access-j82vx\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:43.802779 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.802760    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5cqd\" (UniqueName: \"kubernetes.io/projected/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-kube-api-access-p5cqd\") pod \"periodic-gathering-pntg4-8mc75\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") " pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:36:43.802824 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.802786    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npfv7\" (UniqueName: \"kubernetes.io/projected/4a6021d6-6317-473f-97bc-ea023261d90b-kube-api-access-npfv7\") pod \"node-ca-qkvdd\" (UID: \"4a6021d6-6317-473f-97bc-ea023261d90b\") " pod="openshift-image-registry/node-ca-qkvdd"
Oct 30 18:36:43.802824 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.802790    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pphxn\" (UniqueName: \"kubernetes.io/projected/114fbcb8-1004-410d-98bc-dc60f8fcae3c-kube-api-access-pphxn\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.802898 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.802871    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn988\" (UniqueName: \"kubernetes.io/projected/f98872b6-9c55-467c-9c70-2831de3fb841-kube-api-access-vn988\") pod \"ingress-canary-hwxx2\" (UID: \"f98872b6-9c55-467c-9c70-2831de3fb841\") " pod="openshift-ingress-canary/ingress-canary-hwxx2"
Oct 30 18:36:43.852462 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852433    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-modprobe-d\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.852561 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852465    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-run\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.852561 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852481    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.852561 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852501    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-device-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.852561 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852524    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:43.852561 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852550    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.852561 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852556    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-run\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852567    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852575    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-socket-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852595    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-device-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852617    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852622    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f51455e1-dbba-4186-adbf-60f44c9edbe5-dbus\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:43.852662    4544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852670    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-modprobe-d\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852675    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-run-k8s-cni-cncf-io\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852678    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f51455e1-dbba-4186-adbf-60f44c9edbe5-dbus\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:43.852732    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls podName:11db0a9a-1a94-430c-b395-6ea6bbeba528 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:44.35271847 +0000 UTC m=+1.878593573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6zrww" (UID: "11db0a9a-1a94-430c-b395-6ea6bbeba528") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852707    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-run-k8s-cni-cncf-io\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852664    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-socket-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852754    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-conf-dir\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852779    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-conf-dir\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852798    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-etc-openvswitch\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852824    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-sysconfig\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852834    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-etc-openvswitch\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852850    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-socket-dir-parent\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852873    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-sysconfig\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.852865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852875    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-var-lib-kubelet\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852900    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-etc-selinux\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852920    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-socket-dir-parent\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852929    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab0116fc-0400-4f7d-be91-27804b24c8f4-cnibin\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852977    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-etc-selinux\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852979    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab0116fc-0400-4f7d-be91-27804b24c8f4-cnibin\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852982    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-system-cni-dir\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853010    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-system-cni-dir\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853018    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-run-ovn\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.852983    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-var-lib-kubelet\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853043    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d7ebad18-eccb-47c9-b259-67c6d6b84d0f-hosts-file\") pod \"node-resolver-nmmwh\" (UID: \"d7ebad18-eccb-47c9-b259-67c6d6b84d0f\") " pod="openshift-dns/node-resolver-nmmwh"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853067    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-run-ovn\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853076    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d7ebad18-eccb-47c9-b259-67c6d6b84d0f-hosts-file\") pod \"node-resolver-nmmwh\" (UID: \"d7ebad18-eccb-47c9-b259-67c6d6b84d0f\") " pod="openshift-dns/node-resolver-nmmwh"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853067    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-sysctl-conf\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853111    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-sys-fs\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853127    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-etc-kubernetes\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853142    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-kubernetes\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853156    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-sys-fs\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853158    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f51455e1-dbba-4186-adbf-60f44c9edbe5-kubelet-config\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853181    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f51455e1-dbba-4186-adbf-60f44c9edbe5-kubelet-config\") pod \"global-pull-secret-syncer-b4fj5\" (UID: \"f51455e1-dbba-4186-adbf-60f44c9edbe5\") " pod="kube-system/global-pull-secret-syncer-b4fj5"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853186    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs\") pod \"network-metrics-daemon-7wdcf\" (UID: \"92a21826-d145-4b9b-9897-aaf21f63a67b\") " pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853186    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-sysctl-conf\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853202    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-etc-kubernetes\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853216    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-hostroot\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853228    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-kubernetes\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853241    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-var-lib-openvswitch\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853254    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-hostroot\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853264    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-log-socket\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853285    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-var-lib-openvswitch\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853288    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-log-socket\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853296    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-cni-netd\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853333    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7894bc9-1a77-42e3-a59b-926729c93dc9-host-slash\") pod \"iptables-alerter-nssbw\" (UID: \"f7894bc9-1a77-42e3-a59b-926729c93dc9\") " pod="openshift-network-operator/iptables-alerter-nssbw"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853358    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-kubelet\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853377    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f7894bc9-1a77-42e3-a59b-926729c93dc9-host-slash\") pod \"iptables-alerter-nssbw\" (UID: \"f7894bc9-1a77-42e3-a59b-926729c93dc9\") " pod="openshift-network-operator/iptables-alerter-nssbw"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853381    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-run-netns\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853386    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-cni-netd\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853403    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-kubelet\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853422    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-cni-bin\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853436    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-systemd\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853437    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-run-netns\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853460    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-slash\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853485    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-run-ovn-kubernetes\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853493    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-cni-bin\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853481    4544 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853509    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-lib-modules\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853511    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-slash\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853521    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-systemd\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853527    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-host-run-ovn-kubernetes\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853531    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-host\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853563    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-var-lib-cni-multus\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853566    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-host\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853580    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-run-multus-certs\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853596    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-sys\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853597    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-lib-modules\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853626    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-var-lib-cni-multus\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853630    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853647    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-run-multus-certs\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853656    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-cnibin\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853654    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-sys\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:43.853660    4544 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853680    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-run-netns\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853694    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-cnibin\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853701    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-var-lib-cni-bin\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853716    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-run-netns\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:43.853738    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm podName:11db0a9a-1a94-430c-b395-6ea6bbeba528 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:44.353721695 +0000 UTC m=+1.879596789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6zrww" (UID: "11db0a9a-1a94-430c-b395-6ea6bbeba528") : configmap "kube-rbac-proxy" not found
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853745    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-var-lib-kubelet\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853746    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-var-lib-cni-bin\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853718    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-host-var-lib-kubelet\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.853731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853783    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-systemd-units\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853798    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-run-systemd\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853825    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-run-openvswitch\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853844    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-systemd-units\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853848    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab0116fc-0400-4f7d-be91-27804b24c8f4-system-cni-dir\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853859    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-run-systemd\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853871    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-cni-dir\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853882    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-run-openvswitch\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853895    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-node-log\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853917    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a517bba6-536f-4dac-ba8b-2e71964ef8c5-multus-cni-dir\") pod \"multus-54jpr\" (UID: \"a517bba6-536f-4dac-ba8b-2e71964ef8c5\") " pod="openshift-multus/multus-54jpr"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853921    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/114fbcb8-1004-410d-98bc-dc60f8fcae3c-node-log\") pod \"ovnkube-node-nxnnz\" (UID: \"114fbcb8-1004-410d-98bc-dc60f8fcae3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853909    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab0116fc-0400-4f7d-be91-27804b24c8f4-system-cni-dir\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853926    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-sysctl-d\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853967    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab0116fc-0400-4f7d-be91-27804b24c8f4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853999    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d93ec858-ae33-473a-ad4b-18b88664094c-etc-sysctl-d\") pod \"tuned-78mjc\" (UID: \"d93ec858-ae33-473a-ad4b-18b88664094c\") " pod="openshift-cluster-node-tuning-operator/tuned-78mjc"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.853998    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/11db0a9a-1a94-430c-b395-6ea6bbeba528-crio-socket\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.854023    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.854040    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a6021d6-6317-473f-97bc-ea023261d90b-host\") pod \"node-ca-qkvdd\" (UID: \"4a6021d6-6317-473f-97bc-ea023261d90b\") " pod="openshift-image-registry/node-ca-qkvdd"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.854041    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/11db0a9a-1a94-430c-b395-6ea6bbeba528-crio-socket\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.854041    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab0116fc-0400-4f7d-be91-27804b24c8f4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cxhd5\" (UID: \"ab0116fc-0400-4f7d-be91-27804b24c8f4\") " pod="openshift-multus/multus-additional-cni-plugins-cxhd5"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.854054    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-registration-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.854080    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a6021d6-6317-473f-97bc-ea023261d90b-host\") pod \"node-ca-qkvdd\" (UID: \"4a6021d6-6317-473f-97bc-ea023261d90b\") " pod="openshift-image-registry/node-ca-qkvdd"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.854115    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90c6593d-b40a-43ae-abc0-ce015e48c289-registration-dir\") pod \"aws-ebs-csi-driver-node-wmqtz\" (UID: \"90c6593d-b40a-43ae-abc0-ce015e48c289\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wmqtz"
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:43.854122    4544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:43.854182    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls podName:5db1f514-bc37-424a-89ce-71dbc6a520e9 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:44.354165885 +0000 UTC m=+1.880040980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls") pod "dns-default-zmj6x" (UID: "5db1f514-bc37-424a-89ce-71dbc6a520e9") : secret "dns-default-metrics-tls" not found
Oct 30 18:36:43.855763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.855632    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92a21826-d145-4b9b-9897-aaf21f63a67b-metrics-certs\") pod \"network-metrics-daemon-7wdcf\" (UID: \"92a21826-d145-4b9b-9897-aaf21f63a67b\") " pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:43.947550 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:43.947529    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wdcf"
Oct 30 18:36:44.355320 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:44.355289    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7wdcf"]
Oct 30 18:36:44.359322 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:44.359291    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92a21826_d145_4b9b_9897_aaf21f63a67b.slice/crio-9e9b2299d2dcc7dd7ac8d5678449d7bb70b3e7f7ad6c4d05d21f555ea6e9e0d1 WatchSource:0}: Error finding container 9e9b2299d2dcc7dd7ac8d5678449d7bb70b3e7f7ad6c4d05d21f555ea6e9e0d1: Status 404 returned error can't find the container with id 9e9b2299d2dcc7dd7ac8d5678449d7bb70b3e7f7ad6c4d05d21f555ea6e9e0d1
Oct 30 18:36:44.359525 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:44.359506    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:44.359645 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:44.359543    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:44.359645 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:44.359567    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:44.359757 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:44.359668    4544 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:36:44.359757 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:44.359697    4544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:44.359757 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:44.359698    4544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:36:44.359757 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:44.359732    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm podName:11db0a9a-1a94-430c-b395-6ea6bbeba528 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:45.359715598 +0000 UTC m=+2.885590682 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6zrww" (UID: "11db0a9a-1a94-430c-b395-6ea6bbeba528") : configmap "kube-rbac-proxy" not found
Oct 30 18:36:44.359757 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:44.359749    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls podName:11db0a9a-1a94-430c-b395-6ea6bbeba528 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:45.359743102 +0000 UTC m=+2.885618178 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6zrww" (UID: "11db0a9a-1a94-430c-b395-6ea6bbeba528") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:44.359757 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:44.359759    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls podName:5db1f514-bc37-424a-89ce-71dbc6a520e9 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:45.359754621 +0000 UTC m=+2.885629702 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls") pod "dns-default-zmj6x" (UID: "5db1f514-bc37-424a-89ce-71dbc6a520e9") : secret "dns-default-metrics-tls" not found
Oct 30 18:36:44.361664 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:44.361646    4544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:36:44.637477 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:44.637418    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7wdcf" event={"ID":"92a21826-d145-4b9b-9897-aaf21f63a67b","Type":"ContainerStarted","Data":"9e9b2299d2dcc7dd7ac8d5678449d7bb70b3e7f7ad6c4d05d21f555ea6e9e0d1"}
Oct 30 18:36:45.366706 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:45.366674    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:45.366945 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:45.366713    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:45.366945 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:45.366743    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:45.366945 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:45.366776    4544 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:36:45.366945 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:45.366842    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm podName:11db0a9a-1a94-430c-b395-6ea6bbeba528 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:47.366828142 +0000 UTC m=+4.892703220 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6zrww" (UID: "11db0a9a-1a94-430c-b395-6ea6bbeba528") : configmap "kube-rbac-proxy" not found
Oct 30 18:36:45.366945 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:45.366845    4544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:45.366945 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:45.366908    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls podName:11db0a9a-1a94-430c-b395-6ea6bbeba528 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:47.366893591 +0000 UTC m=+4.892768665 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6zrww" (UID: "11db0a9a-1a94-430c-b395-6ea6bbeba528") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:45.369022 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:45.368993    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5db1f514-bc37-424a-89ce-71dbc6a520e9-metrics-tls\") pod \"dns-default-zmj6x\" (UID: \"5db1f514-bc37-424a-89ce-71dbc6a520e9\") " pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:45.449754 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:45.449726    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:45.452741 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:45.452718    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nmmwh_d7ebad18-eccb-47c9-b259-67c6d6b84d0f/dns-node-resolver/0.log"
Oct 30 18:36:45.828731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:45.828699    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zmj6x"]
Oct 30 18:36:45.832230 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:45.832201    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5db1f514_bc37_424a_89ce_71dbc6a520e9.slice/crio-9f994fc701f9c3cc78b142fa9f64aafa3b294add6fec02eb643300ce9d2e40eb WatchSource:0}: Error finding container 9f994fc701f9c3cc78b142fa9f64aafa3b294add6fec02eb643300ce9d2e40eb: Status 404 returned error can't find the container with id 9f994fc701f9c3cc78b142fa9f64aafa3b294add6fec02eb643300ce9d2e40eb
Oct 30 18:36:46.452706 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:46.452682    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qkvdd_4a6021d6-6317-473f-97bc-ea023261d90b/node-ca/0.log"
Oct 30 18:36:46.644496 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:46.644415    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7wdcf" event={"ID":"92a21826-d145-4b9b-9897-aaf21f63a67b","Type":"ContainerStarted","Data":"b989de6af3256c5ee3c3b56be1010cd7a13259935efbd9362987dd21c9d09b57"}
Oct 30 18:36:46.644496 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:46.644469    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7wdcf" event={"ID":"92a21826-d145-4b9b-9897-aaf21f63a67b","Type":"ContainerStarted","Data":"1475cb119779c6af0ada4f0b00340b6e916fe61c9ba4a7513c20ab7660cf6632"}
Oct 30 18:36:46.645507 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:46.645484    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zmj6x" event={"ID":"5db1f514-bc37-424a-89ce-71dbc6a520e9","Type":"ContainerStarted","Data":"9f994fc701f9c3cc78b142fa9f64aafa3b294add6fec02eb643300ce9d2e40eb"}
Oct 30 18:36:47.381541 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:47.381506    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:47.381986 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:47.381571    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:47.381986 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:47.381730    4544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:47.381986 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:47.381793    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls podName:11db0a9a-1a94-430c-b395-6ea6bbeba528 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:51.381775073 +0000 UTC m=+8.907650152 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6zrww" (UID: "11db0a9a-1a94-430c-b395-6ea6bbeba528") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:47.382195 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:47.382158    4544 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:36:47.382242 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:47.382204    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm podName:11db0a9a-1a94-430c-b395-6ea6bbeba528 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:51.382190008 +0000 UTC m=+8.908065085 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6zrww" (UID: "11db0a9a-1a94-430c-b395-6ea6bbeba528") : configmap "kube-rbac-proxy" not found
Oct 30 18:36:47.453198 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:47.453175    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hwxx2_f98872b6-9c55-467c-9c70-2831de3fb841/serve-healthcheck-canary/0.log"
Oct 30 18:36:47.853455 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:47.853420    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-nr9vn_9afdc202-0860-42d2-99dc-e1e028b75953/migrator/0.log"
Oct 30 18:36:48.653581 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:48.653545    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zmj6x" event={"ID":"5db1f514-bc37-424a-89ce-71dbc6a520e9","Type":"ContainerStarted","Data":"ca08a9b62aacdfe4783daa10fb499dd2cdcfb073b1855ea08fc459be0f56552d"}
Oct 30 18:36:48.653959 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:48.653597    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zmj6x" event={"ID":"5db1f514-bc37-424a-89ce-71dbc6a520e9","Type":"ContainerStarted","Data":"6f69af62b193eb04dc0a5be68eb8a1c684a5fb6716d3c986a055d2c5d77f7fa7"}
Oct 30 18:36:48.653959 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:48.653719    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:49.841191 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:49.841158    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:36:51.408220 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:51.408183    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:51.408727 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:51.408236    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:51.408727 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:51.408351    4544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:51.408727 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:36:51.408422    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls podName:11db0a9a-1a94-430c-b395-6ea6bbeba528 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:59.408404795 +0000 UTC m=+16.934280231 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6zrww" (UID: "11db0a9a-1a94-430c-b395-6ea6bbeba528") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:51.408727 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:51.408694    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11db0a9a-1a94-430c-b395-6ea6bbeba528-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:51.840289 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:51.840262    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:53.007799 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:53.007764    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:36:53.008190 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:53.007945    4544 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:36:53.041197 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:53.041175    4544 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-122.us-west-2.compute.internal" event="NodeReady"
Oct 30 18:36:55.249127 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.249094    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-6wkbq"]
Oct 30 18:36:55.250931 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.250914    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-6wkbq"
Oct 30 18:36:55.301471 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.301448    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 18:36:55.301633 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.301618    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 18:36:55.338672 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.338642    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vcwb\" (UniqueName: \"kubernetes.io/projected/136888e1-bf5a-4a00-ac01-3365853beddc-kube-api-access-9vcwb\") pod \"downloads-6dd789c8b7-6wkbq\" (UID: \"136888e1-bf5a-4a00-ac01-3365853beddc\") " pod="openshift-console/downloads-6dd789c8b7-6wkbq"
Oct 30 18:36:55.338801 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.338695    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/136888e1-bf5a-4a00-ac01-3365853beddc-tmp\") pod \"downloads-6dd789c8b7-6wkbq\" (UID: \"136888e1-bf5a-4a00-ac01-3365853beddc\") " pod="openshift-console/downloads-6dd789c8b7-6wkbq"
Oct 30 18:36:55.341066 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.341046    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-vzt9s\""
Oct 30 18:36:55.348294 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.348267    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-6wkbq"]
Oct 30 18:36:55.348670 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.348653    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d8fc8758d-wq4lx"]
Oct 30 18:36:55.350994 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.350981    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.401727 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.401699    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d8fc8758d-wq4lx"]
Oct 30 18:36:55.439158 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.439129    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8c219ee-00a1-4aaa-8527-b0779be307aa-image-registry-private-configuration\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.439288 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.439201    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8c219ee-00a1-4aaa-8527-b0779be307aa-registry-tls\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.439288 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.439247    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8c219ee-00a1-4aaa-8527-b0779be307aa-registry-certificates\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.439288 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.439264    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8c219ee-00a1-4aaa-8527-b0779be307aa-trusted-ca\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.439406 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.439297    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vcwb\" (UniqueName: \"kubernetes.io/projected/136888e1-bf5a-4a00-ac01-3365853beddc-kube-api-access-9vcwb\") pod \"downloads-6dd789c8b7-6wkbq\" (UID: \"136888e1-bf5a-4a00-ac01-3365853beddc\") " pod="openshift-console/downloads-6dd789c8b7-6wkbq"
Oct 30 18:36:55.439406 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.439394    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/136888e1-bf5a-4a00-ac01-3365853beddc-tmp\") pod \"downloads-6dd789c8b7-6wkbq\" (UID: \"136888e1-bf5a-4a00-ac01-3365853beddc\") " pod="openshift-console/downloads-6dd789c8b7-6wkbq"
Oct 30 18:36:55.439470 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.439415    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8c219ee-00a1-4aaa-8527-b0779be307aa-ca-trust-extracted\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.439470 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.439447    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8c219ee-00a1-4aaa-8527-b0779be307aa-bound-sa-token\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.439560 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.439471    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8c219ee-00a1-4aaa-8527-b0779be307aa-installation-pull-secrets\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.439560 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.439497    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btkvd\" (UniqueName: \"kubernetes.io/projected/e8c219ee-00a1-4aaa-8527-b0779be307aa-kube-api-access-btkvd\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.439859 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.439843    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/136888e1-bf5a-4a00-ac01-3365853beddc-tmp\") pod \"downloads-6dd789c8b7-6wkbq\" (UID: \"136888e1-bf5a-4a00-ac01-3365853beddc\") " pod="openshift-console/downloads-6dd789c8b7-6wkbq"
Oct 30 18:36:55.447772 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.447749    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-h758x\""
Oct 30 18:36:55.448171 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.448137    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 18:36:55.448171 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.448162    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 18:36:55.448350 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.448156    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 18:36:55.448412 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.448369    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 18:36:55.496904 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.496875    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vcwb\" (UniqueName: \"kubernetes.io/projected/136888e1-bf5a-4a00-ac01-3365853beddc-kube-api-access-9vcwb\") pod \"downloads-6dd789c8b7-6wkbq\" (UID: \"136888e1-bf5a-4a00-ac01-3365853beddc\") " pod="openshift-console/downloads-6dd789c8b7-6wkbq"
Oct 30 18:36:55.540484 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.540461    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8c219ee-00a1-4aaa-8527-b0779be307aa-registry-certificates\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.540595 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.540498    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8c219ee-00a1-4aaa-8527-b0779be307aa-trusted-ca\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.540595 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.540542    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8c219ee-00a1-4aaa-8527-b0779be307aa-ca-trust-extracted\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.540595 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.540557    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8c219ee-00a1-4aaa-8527-b0779be307aa-bound-sa-token\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.540595 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.540575    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8c219ee-00a1-4aaa-8527-b0779be307aa-installation-pull-secrets\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.540595 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.540589    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btkvd\" (UniqueName: \"kubernetes.io/projected/e8c219ee-00a1-4aaa-8527-b0779be307aa-kube-api-access-btkvd\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.540851 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.540633    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8c219ee-00a1-4aaa-8527-b0779be307aa-image-registry-private-configuration\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.540851 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.540662    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8c219ee-00a1-4aaa-8527-b0779be307aa-registry-tls\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.540992 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.540969    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8c219ee-00a1-4aaa-8527-b0779be307aa-ca-trust-extracted\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.541703 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.541676    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8c219ee-00a1-4aaa-8527-b0779be307aa-registry-certificates\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.541901 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.541880    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8c219ee-00a1-4aaa-8527-b0779be307aa-trusted-ca\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.543011 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.542991    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8c219ee-00a1-4aaa-8527-b0779be307aa-installation-pull-secrets\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.543110 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.543097    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8c219ee-00a1-4aaa-8527-b0779be307aa-image-registry-private-configuration\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.543341 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.543321    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8c219ee-00a1-4aaa-8527-b0779be307aa-registry-tls\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.561005 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.560985    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-6wkbq"
Oct 30 18:36:55.594180 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.594150    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8c219ee-00a1-4aaa-8527-b0779be307aa-bound-sa-token\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.594486 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.594464    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btkvd\" (UniqueName: \"kubernetes.io/projected/e8c219ee-00a1-4aaa-8527-b0779be307aa-kube-api-access-btkvd\") pod \"image-registry-7d8fc8758d-wq4lx\" (UID: \"e8c219ee-00a1-4aaa-8527-b0779be307aa\") " pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.660229 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.660202    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:55.967016 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:55.966938    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-6wkbq"]
Oct 30 18:36:55.970465 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:55.970439    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod136888e1_bf5a_4a00_ac01_3365853beddc.slice/crio-3fce131b85f8c035401a21f75a3608fd61d380a17c66f5b8a5246da1b307f415 WatchSource:0}: Error finding container 3fce131b85f8c035401a21f75a3608fd61d380a17c66f5b8a5246da1b307f415: Status 404 returned error can't find the container with id 3fce131b85f8c035401a21f75a3608fd61d380a17c66f5b8a5246da1b307f415
Oct 30 18:36:56.023393 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:56.023369    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d8fc8758d-wq4lx"]
Oct 30 18:36:56.026777 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:56.026748    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c219ee_00a1_4aaa_8527_b0779be307aa.slice/crio-30d78ea659faf40c35c32fdadbe8e9811f8650d7bfe570228bd8aabe542159e7 WatchSource:0}: Error finding container 30d78ea659faf40c35c32fdadbe8e9811f8650d7bfe570228bd8aabe542159e7: Status 404 returned error can't find the container with id 30d78ea659faf40c35c32fdadbe8e9811f8650d7bfe570228bd8aabe542159e7
Oct 30 18:36:56.678525 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:56.678484    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx" event={"ID":"e8c219ee-00a1-4aaa-8527-b0779be307aa","Type":"ContainerStarted","Data":"877313b968663a08e2224e5f385588bc5f82e9d704761e2403a08ba40430f460"}
Oct 30 18:36:56.678966 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:56.678534    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx" event={"ID":"e8c219ee-00a1-4aaa-8527-b0779be307aa","Type":"ContainerStarted","Data":"30d78ea659faf40c35c32fdadbe8e9811f8650d7bfe570228bd8aabe542159e7"}
Oct 30 18:36:56.678966 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:56.678559    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:36:56.679578 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:56.679557    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-6wkbq" event={"ID":"136888e1-bf5a-4a00-ac01-3365853beddc","Type":"ContainerStarted","Data":"3fce131b85f8c035401a21f75a3608fd61d380a17c66f5b8a5246da1b307f415"}
Oct 30 18:36:56.789045 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:56.788984    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx" podStartSLOduration=1.7889655709999999 podStartE2EDuration="1.788965571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:56.788477586 +0000 UTC m=+14.314352680" watchObservedRunningTime="2026-10-30 18:36:56.788965571 +0000 UTC m=+14.314840667"
Oct 30 18:36:58.659432 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:58.659402    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zmj6x"
Oct 30 18:36:59.473105 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:59.473060    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:59.475728 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:59.475697    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11db0a9a-1a94-430c-b395-6ea6bbeba528-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6zrww\" (UID: \"11db0a9a-1a94-430c-b395-6ea6bbeba528\") " pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:59.551096 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:59.551061    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6zrww"
Oct 30 18:36:59.918557 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:36:59.918518    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6zrww"]
Oct 30 18:36:59.954968 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:36:59.954933    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11db0a9a_1a94_430c_b395_6ea6bbeba528.slice/crio-c60697189524df72ffc9f5a391bda17cf2657b0de02bbe26e0a7c631653a9b28 WatchSource:0}: Error finding container c60697189524df72ffc9f5a391bda17cf2657b0de02bbe26e0a7c631653a9b28: Status 404 returned error can't find the container with id c60697189524df72ffc9f5a391bda17cf2657b0de02bbe26e0a7c631653a9b28
Oct 30 18:37:00.695478 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:00.695444    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6zrww" event={"ID":"11db0a9a-1a94-430c-b395-6ea6bbeba528","Type":"ContainerStarted","Data":"cce0efc7028764c36d97e744e6aca98d8fa417791ce6402f148130dd8ecbaf0f"}
Oct 30 18:37:00.695688 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:00.695512    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6zrww" event={"ID":"11db0a9a-1a94-430c-b395-6ea6bbeba528","Type":"ContainerStarted","Data":"c60697189524df72ffc9f5a391bda17cf2657b0de02bbe26e0a7c631653a9b28"}
Oct 30 18:37:01.702806 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:01.702768    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6zrww" event={"ID":"11db0a9a-1a94-430c-b395-6ea6bbeba528","Type":"ContainerStarted","Data":"687b4d6362fa217c86fd4bc389af6fb569feb3485d87f4d814206fb54554437d"}
Oct 30 18:37:03.717156 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:03.717114    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6zrww" event={"ID":"11db0a9a-1a94-430c-b395-6ea6bbeba528","Type":"ContainerStarted","Data":"1d54c4492af9220f794460b17eff7fbd4ad871add005a56f5b51069815f217e8"}
Oct 30 18:37:05.182507 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.182471    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb"]
Oct 30 18:37:05.184536 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.184519    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb"
Oct 30 18:37:05.213400 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.213370    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4cbe37d7-4720-46d0-9beb-f49960526365-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-p4gdb\" (UID: \"4cbe37d7-4720-46d0-9beb-f49960526365\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb"
Oct 30 18:37:05.213523 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.213404    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4cbe37d7-4720-46d0-9beb-f49960526365-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-p4gdb\" (UID: \"4cbe37d7-4720-46d0-9beb-f49960526365\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb"
Oct 30 18:37:05.213523 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.213424    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9ckh\" (UniqueName: \"kubernetes.io/projected/4cbe37d7-4720-46d0-9beb-f49960526365-kube-api-access-j9ckh\") pod \"prometheus-operator-9cd89b74c-p4gdb\" (UID: \"4cbe37d7-4720-46d0-9beb-f49960526365\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb"
Oct 30 18:37:05.213618 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.213554    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4cbe37d7-4720-46d0-9beb-f49960526365-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-p4gdb\" (UID: \"4cbe37d7-4720-46d0-9beb-f49960526365\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb"
Oct 30 18:37:05.233872 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.233847    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 18:37:05.279715 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.279679    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:37:05.279715 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.279686    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:37:05.280088 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.280069    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-q26b5\""
Oct 30 18:37:05.280189 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.280087    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 18:37:05.280189 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.280177    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 18:37:05.314581 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.314543    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4cbe37d7-4720-46d0-9beb-f49960526365-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-p4gdb\" (UID: \"4cbe37d7-4720-46d0-9beb-f49960526365\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb"
Oct 30 18:37:05.314581 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.314578    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4cbe37d7-4720-46d0-9beb-f49960526365-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-p4gdb\" (UID: \"4cbe37d7-4720-46d0-9beb-f49960526365\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb"
Oct 30 18:37:05.314783 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.314599    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9ckh\" (UniqueName: \"kubernetes.io/projected/4cbe37d7-4720-46d0-9beb-f49960526365-kube-api-access-j9ckh\") pod \"prometheus-operator-9cd89b74c-p4gdb\" (UID: \"4cbe37d7-4720-46d0-9beb-f49960526365\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb"
Oct 30 18:37:05.314783 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.314650    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4cbe37d7-4720-46d0-9beb-f49960526365-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-p4gdb\" (UID: \"4cbe37d7-4720-46d0-9beb-f49960526365\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb"
Oct 30 18:37:05.315688 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.315639    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4cbe37d7-4720-46d0-9beb-f49960526365-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-p4gdb\" (UID: \"4cbe37d7-4720-46d0-9beb-f49960526365\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb"
Oct 30 18:37:05.317314 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.317286    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4cbe37d7-4720-46d0-9beb-f49960526365-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-p4gdb\" (UID: \"4cbe37d7-4720-46d0-9beb-f49960526365\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb"
Oct 30 18:37:05.317451 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.317424    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4cbe37d7-4720-46d0-9beb-f49960526365-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-p4gdb\" (UID: \"4cbe37d7-4720-46d0-9beb-f49960526365\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb"
Oct 30 18:37:05.326874 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.326854    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb"]
Oct 30 18:37:05.372159 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.372133    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9ckh\" (UniqueName: \"kubernetes.io/projected/4cbe37d7-4720-46d0-9beb-f49960526365-kube-api-access-j9ckh\") pod \"prometheus-operator-9cd89b74c-p4gdb\" (UID: \"4cbe37d7-4720-46d0-9beb-f49960526365\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb"
Oct 30 18:37:05.496998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.496917    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb"
Oct 30 18:37:05.540485 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:37:05.540444    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:37:05.868900 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:05.868827    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb"]
Oct 30 18:37:05.872549 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:37:05.872520    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cbe37d7_4720_46d0_9beb_f49960526365.slice/crio-ae5f324a4279e13672c9935b87ec46e8a7c253638afc5fcf2d6205c58ab09863 WatchSource:0}: Error finding container ae5f324a4279e13672c9935b87ec46e8a7c253638afc5fcf2d6205c58ab09863: Status 404 returned error can't find the container with id ae5f324a4279e13672c9935b87ec46e8a7c253638afc5fcf2d6205c58ab09863
Oct 30 18:37:06.613625 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:06.613585    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxnnz"
Oct 30 18:37:06.729561 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:06.729514    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb" event={"ID":"4cbe37d7-4720-46d0-9beb-f49960526365","Type":"ContainerStarted","Data":"ae5f324a4279e13672c9935b87ec46e8a7c253638afc5fcf2d6205c58ab09863"}
Oct 30 18:37:07.735205 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:07.735178    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb" event={"ID":"4cbe37d7-4720-46d0-9beb-f49960526365","Type":"ContainerStarted","Data":"b1f0dea22fb14658e15f0172fc0bd51bd3c6549a1b2ec54d6e02df63f39b67d5"}
Oct 30 18:37:08.741252 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:08.741217    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb" event={"ID":"4cbe37d7-4720-46d0-9beb-f49960526365","Type":"ContainerStarted","Data":"23bc6248a8eb350db8b09fd5f22b9476e6d7c3d7d5897c634eadc241a856c747"}
Oct 30 18:37:08.850570 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:08.850283    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-p4gdb" podStartSLOduration=2.11227509 podStartE2EDuration="3.850267117s" totalImagesPullingTime="1.737992027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:08.849982933 +0000 UTC m=+26.375858028" watchObservedRunningTime="2026-10-30 18:37:08.850267117 +0000 UTC m=+26.376142221"
Oct 30 18:37:09.844229 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:09.844200    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qk8z2"
Oct 30 18:37:11.869158 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:11.869127    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sgqww"]
Oct 30 18:37:11.886038 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:11.886015    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:11.962563 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:11.962535    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4frv9\" (UniqueName: \"kubernetes.io/projected/a25f2140-3a18-44d6-9cfa-7728f9284fee-kube-api-access-4frv9\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:11.962752 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:11.962598    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a25f2140-3a18-44d6-9cfa-7728f9284fee-root\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:11.962752 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:11.962739    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a25f2140-3a18-44d6-9cfa-7728f9284fee-sys\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:11.962872 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:11.962784    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a25f2140-3a18-44d6-9cfa-7728f9284fee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:11.962872 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:11.962815    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a25f2140-3a18-44d6-9cfa-7728f9284fee-node-exporter-accelerators-collector-config\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:11.962978 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:11.962901    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a25f2140-3a18-44d6-9cfa-7728f9284fee-node-exporter-textfile\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:11.962978 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:11.962929    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a25f2140-3a18-44d6-9cfa-7728f9284fee-metrics-client-ca\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:11.962978 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:11.962956    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a25f2140-3a18-44d6-9cfa-7728f9284fee-node-exporter-wtmp\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:11.963126 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:11.963055    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a25f2140-3a18-44d6-9cfa-7728f9284fee-node-exporter-tls\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.040929 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.040901    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zw6ts\""
Oct 30 18:37:12.040929 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.040919    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 18:37:12.041190 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.041168    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 18:37:12.041266 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.041225    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 18:37:12.063635 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.063594    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a25f2140-3a18-44d6-9cfa-7728f9284fee-root\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.063799 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.063659    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a25f2140-3a18-44d6-9cfa-7728f9284fee-sys\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.063799 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.063684    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a25f2140-3a18-44d6-9cfa-7728f9284fee-root\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.063799 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.063694    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a25f2140-3a18-44d6-9cfa-7728f9284fee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.063799 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.063720    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a25f2140-3a18-44d6-9cfa-7728f9284fee-node-exporter-accelerators-collector-config\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.063799 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.063753    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a25f2140-3a18-44d6-9cfa-7728f9284fee-node-exporter-textfile\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.063799 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.063775    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a25f2140-3a18-44d6-9cfa-7728f9284fee-metrics-client-ca\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.063799 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.063774    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a25f2140-3a18-44d6-9cfa-7728f9284fee-sys\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.064125 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.063828    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a25f2140-3a18-44d6-9cfa-7728f9284fee-node-exporter-wtmp\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.064125 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.063884    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a25f2140-3a18-44d6-9cfa-7728f9284fee-node-exporter-tls\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.064125 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.063911    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4frv9\" (UniqueName: \"kubernetes.io/projected/a25f2140-3a18-44d6-9cfa-7728f9284fee-kube-api-access-4frv9\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.064269 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.064205    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a25f2140-3a18-44d6-9cfa-7728f9284fee-node-exporter-wtmp\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.064395 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.064376    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a25f2140-3a18-44d6-9cfa-7728f9284fee-node-exporter-textfile\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.064700 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.064645    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a25f2140-3a18-44d6-9cfa-7728f9284fee-node-exporter-accelerators-collector-config\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.064814 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.064753    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a25f2140-3a18-44d6-9cfa-7728f9284fee-metrics-client-ca\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.066293 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.066272    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a25f2140-3a18-44d6-9cfa-7728f9284fee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.066778 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.066759    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a25f2140-3a18-44d6-9cfa-7728f9284fee-node-exporter-tls\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.119288 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.119230    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4frv9\" (UniqueName: \"kubernetes.io/projected/a25f2140-3a18-44d6-9cfa-7728f9284fee-kube-api-access-4frv9\") pod \"node-exporter-sgqww\" (UID: \"a25f2140-3a18-44d6-9cfa-7728f9284fee\") " pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.226093 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.226062    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sgqww"
Oct 30 18:37:12.256446 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:37:12.256411    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25f2140_3a18_44d6_9cfa_7728f9284fee.slice/crio-47a7ae69a51b56b7bbe38be275b8ede2e06f7c8dc727f0bf15399badb055ef71 WatchSource:0}: Error finding container 47a7ae69a51b56b7bbe38be275b8ede2e06f7c8dc727f0bf15399badb055ef71: Status 404 returned error can't find the container with id 47a7ae69a51b56b7bbe38be275b8ede2e06f7c8dc727f0bf15399badb055ef71
Oct 30 18:37:12.623712 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.623677    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-r52tw"
Oct 30 18:37:12.755091 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:12.755044    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sgqww" event={"ID":"a25f2140-3a18-44d6-9cfa-7728f9284fee","Type":"ContainerStarted","Data":"47a7ae69a51b56b7bbe38be275b8ede2e06f7c8dc727f0bf15399badb055ef71"}
Oct 30 18:37:13.761827 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:13.761796    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-6wkbq" event={"ID":"136888e1-bf5a-4a00-ac01-3365853beddc","Type":"ContainerStarted","Data":"f66a7cdf5641b7917619e3f423108e15c22ce3998cdaa4313c811025115d87c5"}
Oct 30 18:37:13.762284 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:13.761980    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-6wkbq"
Oct 30 18:37:13.763277 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:13.763253    4544 patch_prober.go:29] interesting pod/downloads-6dd789c8b7-6wkbq container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.11:8080/\": dial tcp 10.133.0.11:8080: connect: connection refused" start-of-body=
Oct 30 18:37:13.763377 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:13.763296    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-6dd789c8b7-6wkbq" podUID="136888e1-bf5a-4a00-ac01-3365853beddc" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.11:8080/\": dial tcp 10.133.0.11:8080: connect: connection refused"
Oct 30 18:37:13.870943 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:13.870856    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-6wkbq" podStartSLOduration=1.323990644 podStartE2EDuration="18.870841242s" totalImagesPullingTime="17.546850598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:13.869926617 +0000 UTC m=+31.395801714" watchObservedRunningTime="2026-10-30 18:37:13.870841242 +0000 UTC m=+31.396716339"
Oct 30 18:37:14.767824 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:14.767778    4544 generic.go:356] "Generic (PLEG): container finished" podID="a25f2140-3a18-44d6-9cfa-7728f9284fee" containerID="4f38bcd21a88ddf3b72abf4ab8dd013165c97a1c697b6e4cf9d3088be304be65" exitCode=0
Oct 30 18:37:14.768318 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:14.767870    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sgqww" event={"ID":"a25f2140-3a18-44d6-9cfa-7728f9284fee","Type":"ContainerDied","Data":"4f38bcd21a88ddf3b72abf4ab8dd013165c97a1c697b6e4cf9d3088be304be65"}
Oct 30 18:37:14.782008 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:14.781982    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-6wkbq"
Oct 30 18:37:15.774654 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:15.774595    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sgqww" event={"ID":"a25f2140-3a18-44d6-9cfa-7728f9284fee","Type":"ContainerStarted","Data":"e203317a9c1fb264911308f8ffee5827fa833c48afe95ab87916481bca78df3a"}
Oct 30 18:37:15.775119 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:15.774682    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sgqww" event={"ID":"a25f2140-3a18-44d6-9cfa-7728f9284fee","Type":"ContainerStarted","Data":"3a178da807ea3061d4bc59290b1d58bce39b29921031e4033edab3cf0b129fc2"}
Oct 30 18:37:15.891673 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:15.891593    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sgqww" podStartSLOduration=3.190592258 podStartE2EDuration="4.891576889s" totalImagesPullingTime="1.620376741s" totalInitContainerRuntime="80.60789ms" isStatefulPod=true podCreationTimestamp="2026-10-30 18:37:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:15.891534102 +0000 UTC m=+33.417409190" watchObservedRunningTime="2026-10-30 18:37:15.891576889 +0000 UTC m=+33.417451989"
Oct 30 18:37:17.687676 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:17.687645    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d8fc8758d-wq4lx"
Oct 30 18:37:21.142993 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.142955    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:37:21.164067 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.164035    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.201882 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.201850    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:37:21.236361 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.236330    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.236530 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.236371    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.236530 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.236410    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.236743 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.236525    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.236743 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.236655    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.236743 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.236695    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.236857 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.236762    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-web-config\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.236857 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.236800    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.236857 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.236835    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0026cf83-5ea2-4ccd-a916-83763f467e6b-config-out\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.237004 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.236902    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.237004 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.236956    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0026cf83-5ea2-4ccd-a916-83763f467e6b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.237111 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.237016    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.237111 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.237039    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.237111 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.237068    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.237111 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.237096    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.237277 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.237129    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.237277 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.237150    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.237277 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.237165    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.237277 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.237194    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-config\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.237277 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.237208    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6gzx\" (UniqueName: \"kubernetes.io/projected/0026cf83-5ea2-4ccd-a916-83763f467e6b-kube-api-access-s6gzx\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.246836 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.246813    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 18:37:21.246836 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.246814    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 18:37:21.247006 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.246823    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 18:37:21.247006 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.246940    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 18:37:21.247006 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.246949    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 18:37:21.247148 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.247020    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 18:37:21.247148 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.247054    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 18:37:21.247148 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.247062    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5ccr3ko96fg85\""
Oct 30 18:37:21.315826 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.315804    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 18:37:21.316002 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.315978    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rs5tw\""
Oct 30 18:37:21.316002 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.315825    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 18:37:21.316002 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.315910    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 18:37:21.316271 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.315826    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 18:37:21.316271 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.316048    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 18:37:21.338266 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338236    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.338401 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338284    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0026cf83-5ea2-4ccd-a916-83763f467e6b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.338401 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338307    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.338401 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338331    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.338401 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338362    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.338401 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338389    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.338650 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338420    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.338650 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338442    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.338650 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338463    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.338650 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338499    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-config\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.338650 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338521    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6gzx\" (UniqueName: \"kubernetes.io/projected/0026cf83-5ea2-4ccd-a916-83763f467e6b-kube-api-access-s6gzx\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.338650 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338557    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.338650 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338572    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.338650 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338595    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.338650 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338634    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.338650 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338654    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.339053 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338686    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.339053 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338711    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-web-config\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.339053 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338728    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.339053 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.338750    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0026cf83-5ea2-4ccd-a916-83763f467e6b-config-out\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.340013 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.339985    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.340278 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.340255    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.340956 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.340930    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.342494 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.342468    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.343650 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.343267    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.344215 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.344199    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-config\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.344994 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.344970    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.345367 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.345343    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.345492 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.345471    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-web-config\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.346305 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.346247    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.346305 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.346248    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0026cf83-5ea2-4ccd-a916-83763f467e6b-config-out\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.346644 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.346495    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.346852 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.346828    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0026cf83-5ea2-4ccd-a916-83763f467e6b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.347012 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.346993    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.347949 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.347926    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.350036 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.350015    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.362969 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.362949    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 18:37:21.377833 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.377804    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.408812 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.408755    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 18:37:21.408927 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.408893    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 18:37:21.412386 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.412361    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6gzx\" (UniqueName: \"kubernetes.io/projected/0026cf83-5ea2-4ccd-a916-83763f467e6b-kube-api-access-s6gzx\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.412555 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.412536    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.413995 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.413973    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:21.592928 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:21.592890    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:22.111246 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:22.111213    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:37:22.115257 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:37:22.115228    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0026cf83_5ea2_4ccd_a916_83763f467e6b.slice/crio-cd9a84e38eecb1846e9f6124e298d4780c8ad35c2573655fd121466040b8c97c WatchSource:0}: Error finding container cd9a84e38eecb1846e9f6124e298d4780c8ad35c2573655fd121466040b8c97c: Status 404 returned error can't find the container with id cd9a84e38eecb1846e9f6124e298d4780c8ad35c2573655fd121466040b8c97c
Oct 30 18:37:22.800665 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:22.800632    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0026cf83-5ea2-4ccd-a916-83763f467e6b","Type":"ContainerStarted","Data":"cd9a84e38eecb1846e9f6124e298d4780c8ad35c2573655fd121466040b8c97c"}
Oct 30 18:37:24.810506 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:24.810470    4544 generic.go:356] "Generic (PLEG): container finished" podID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerID="691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812" exitCode=0
Oct 30 18:37:24.810934 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:24.810558    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0026cf83-5ea2-4ccd-a916-83763f467e6b","Type":"ContainerDied","Data":"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812"}
Oct 30 18:37:29.829275 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:29.829240    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0026cf83-5ea2-4ccd-a916-83763f467e6b","Type":"ContainerStarted","Data":"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a"}
Oct 30 18:37:29.829727 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:29.829333    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0026cf83-5ea2-4ccd-a916-83763f467e6b","Type":"ContainerStarted","Data":"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3"}
Oct 30 18:37:31.839841 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:31.839794    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0026cf83-5ea2-4ccd-a916-83763f467e6b","Type":"ContainerStarted","Data":"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1"}
Oct 30 18:37:32.846743 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:32.846712    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0026cf83-5ea2-4ccd-a916-83763f467e6b","Type":"ContainerStarted","Data":"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80"}
Oct 30 18:37:32.847185 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:32.846778    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0026cf83-5ea2-4ccd-a916-83763f467e6b","Type":"ContainerStarted","Data":"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590"}
Oct 30 18:37:32.847185 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:32.846804    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0026cf83-5ea2-4ccd-a916-83763f467e6b","Type":"ContainerStarted","Data":"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de"}
Oct 30 18:37:33.105230 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:33.105139    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.490040892 podStartE2EDuration="12.105126059s" totalImagesPullingTime="8.593116415s" totalInitContainerRuntime="21.968752ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:33.05844597 +0000 UTC m=+50.584321066" watchObservedRunningTime="2026-10-30 18:37:33.105126059 +0000 UTC m=+50.631001168"
Oct 30 18:37:35.530498 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:37:35.530464    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:37:36.592975 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:36.592943    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:48.918043 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:48.918016    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:48.982804 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:48.982779    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:37:49.916108 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:37:49.916084    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:05.531372 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:38:05.531331    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:38:05.953767 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:05.953688    4544 generic.go:356] "Generic (PLEG): container finished" podID="a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3" containerID="f4ce3e1ab0b44b64afcdeb40f40b6839ebbd4563df941c82da0a4e0a4380e6b1" exitCode=0
Oct 30 18:38:05.953913 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:05.953766    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-pntg4-8mc75" event={"ID":"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3","Type":"ContainerDied","Data":"f4ce3e1ab0b44b64afcdeb40f40b6839ebbd4563df941c82da0a4e0a4380e6b1"}
Oct 30 18:38:06.382414 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:06.382383    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-pntg4-8mc75"]
Oct 30 18:38:07.085493 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:07.085472    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:38:07.180793 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:07.180767    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-service-ca-bundle\") pod \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") "
Oct 30 18:38:07.180917 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:07.180802    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-kube-api-access-p5cqd\" (UniqueName: \"kubernetes.io/projected/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-kube-api-access-p5cqd\") pod \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") "
Oct 30 18:38:07.180917 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:07.180859    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-archives-path\") pod \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") "
Oct 30 18:38:07.180917 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:07.180884    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-trusted-ca-bundle\") pod \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\" (UID: \"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3\") "
Oct 30 18:38:07.181225 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:07.181194    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-archives-path" pod "a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3" (UID: "a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:07.181341 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:07.181207    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-service-ca-bundle" pod "a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3" (UID: "a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:38:07.181341 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:07.181263    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-trusted-ca-bundle" pod "a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3" (UID: "a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:38:07.183038 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:07.183010    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-kube-api-access-p5cqd" pod "a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3" (UID: "a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3"). InnerVolumeSpecName "kube-api-access-p5cqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:38:07.281305 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:07.281280    4544 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-archives-path\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:07.281305 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:07.281305    4544 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-trusted-ca-bundle\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:07.281464 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:07.281314    4544 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-service-ca-bundle\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:07.281464 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:07.281324    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5cqd\" (UniqueName: \"kubernetes.io/projected/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3-kube-api-access-p5cqd\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:07.961383 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:07.961346    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-pntg4-8mc75" event={"ID":"a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3","Type":"ContainerDied","Data":"f77f8483d1e74bc955cbbc2ba67ad372126d051afb298b5f24534ac009886dd7"}
Oct 30 18:38:07.961383 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:07.961376    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-pntg4-8mc75"
Oct 30 18:38:07.961576 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:07.961394    4544 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f77f8483d1e74bc955cbbc2ba67ad372126d051afb298b5f24534ac009886dd7"
Oct 30 18:38:08.179068 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:08.179041    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-pntg4-8mc75"]
Oct 30 18:38:08.184618 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:08.184580    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-pntg4-8mc75"]
Oct 30 18:38:08.599175 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:08.599144    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3" path="/var/lib/kubelet/pods/a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3/volumes"
Oct 30 18:38:14.264086 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.264048    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:38:14.264692 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.264552    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="prometheus" containerID="cri-o://cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3" gracePeriod=600
Oct 30 18:38:14.264692 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.264569    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="kube-rbac-proxy-web" containerID="cri-o://8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de" gracePeriod=600
Oct 30 18:38:14.264692 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.264582    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="thanos-sidecar" containerID="cri-o://a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1" gracePeriod=600
Oct 30 18:38:14.265064 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.264627    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="config-reloader" containerID="cri-o://a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a" gracePeriod=600
Oct 30 18:38:14.265275 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.264633    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="kube-rbac-proxy" containerID="cri-o://701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590" gracePeriod=600
Oct 30 18:38:14.265275 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.265183    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="kube-rbac-proxy-thanos" containerID="cri-o://980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80" gracePeriod=600
Oct 30 18:38:14.507671 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.507649    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:14.528556 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.528535    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-trusted-ca-bundle\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.528699 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.528675    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-db\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.528763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.528717    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-grpc-tls\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.528763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.528752    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-0\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.528875 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.528781    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-config\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-config\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.528875 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.528816    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-metrics-client-certs\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.528875 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.528833    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-trusted-ca-bundle" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:38:14.528875 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.528850    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0026cf83-5ea2-4ccd-a916-83763f467e6b-tls-assets\" (UniqueName: \"kubernetes.io/projected/0026cf83-5ea2-4ccd-a916-83763f467e6b-tls-assets\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.529074 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.528902    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-1\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.529074 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.528941    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0026cf83-5ea2-4ccd-a916-83763f467e6b-kube-api-access-s6gzx\" (UniqueName: \"kubernetes.io/projected/0026cf83-5ea2-4ccd-a916-83763f467e6b-kube-api-access-s6gzx\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.529074 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.528973    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-tls\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.529074 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.529006    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.529074 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.529040    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-serving-certs-ca-bundle\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.529318 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.529074    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-web-config\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-web-config\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.529318 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.529104    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-2\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.529318 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.529140    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0026cf83-5ea2-4ccd-a916-83763f467e6b-config-out\" (UniqueName: \"kubernetes.io/empty-dir/0026cf83-5ea2-4ccd-a916-83763f467e6b-config-out\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.529318 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.529176    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-thanos-prometheus-http-client-file\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.529318 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.529212    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-kubelet-serving-ca-bundle\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.529318 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.529229    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-1" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:38:14.529638 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.529418    4544 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.529638 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.529436    4544 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.529638 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.529584    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-serving-certs-ca-bundle" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:38:14.529961 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.529826    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-kubelet-serving-ca-bundle" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:38:14.530085 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.529979    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-2" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:38:14.530156 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.530045    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-db" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:14.531107 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.531064    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-0" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:38:14.532209 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.532179    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-config" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:38:14.532209 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.532199    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0026cf83-5ea2-4ccd-a916-83763f467e6b-tls-assets" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:38:14.532381 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.532209    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-tls" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:38:14.532443 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.532423    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-thanos-prometheus-http-client-file" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:38:14.532562 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.532537    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-kube-rbac-proxy-web" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:38:14.532711 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.532694    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-grpc-tls" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:38:14.533116 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.533089    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0026cf83-5ea2-4ccd-a916-83763f467e6b-kube-api-access-s6gzx" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "kube-api-access-s6gzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:38:14.533227 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.533205    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-metrics-client-certs" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:38:14.533561 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.533541    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0026cf83-5ea2-4ccd-a916-83763f467e6b-config-out" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:14.541642 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.541621    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-web-config" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:38:14.629985 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.629956    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-metrics-client-ca\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.630124 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.630002    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.630124 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.630040    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-kube-rbac-proxy\") pod \"0026cf83-5ea2-4ccd-a916-83763f467e6b\" (UID: \"0026cf83-5ea2-4ccd-a916-83763f467e6b\") "
Oct 30 18:38:14.630970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.630310    4544 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.630970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.630322    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-metrics-client-ca" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:38:14.630970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.630352    4544 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.630970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.630387    4544 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-db\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.630970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.630404    4544 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-grpc-tls\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.630970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.630419    4544 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.630970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.630433    4544 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-config\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.630970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.630448    4544 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-metrics-client-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.630970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.630461    4544 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0026cf83-5ea2-4ccd-a916-83763f467e6b-tls-assets\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.630970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.630496    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6gzx\" (UniqueName: \"kubernetes.io/projected/0026cf83-5ea2-4ccd-a916-83763f467e6b-kube-api-access-s6gzx\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.630970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.630515    4544 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.630970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.630529    4544 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.630970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.630544    4544 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.630970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.630558    4544 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-web-config\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.630970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.630698    4544 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.630970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.630731    4544 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0026cf83-5ea2-4ccd-a916-83763f467e6b-config-out\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.632457 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.632431    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-kube-rbac-proxy" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:38:14.632834 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.632815    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-thanos-sidecar-tls" pod "0026cf83-5ea2-4ccd-a916-83763f467e6b" (UID: "0026cf83-5ea2-4ccd-a916-83763f467e6b"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:38:14.732044 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.732005    4544 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-kube-rbac-proxy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.732044 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.732041    4544 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0026cf83-5ea2-4ccd-a916-83763f467e6b-configmap-metrics-client-ca\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.732044 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.732052    4544 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0026cf83-5ea2-4ccd-a916-83763f467e6b-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:38:14.985058 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.984981    4544 generic.go:356] "Generic (PLEG): container finished" podID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerID="980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80" exitCode=0
Oct 30 18:38:14.985058 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.985005    4544 generic.go:356] "Generic (PLEG): container finished" podID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerID="701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590" exitCode=0
Oct 30 18:38:14.985058 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.985011    4544 generic.go:356] "Generic (PLEG): container finished" podID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerID="8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de" exitCode=0
Oct 30 18:38:14.985058 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.985017    4544 generic.go:356] "Generic (PLEG): container finished" podID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerID="a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1" exitCode=0
Oct 30 18:38:14.985058 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.985022    4544 generic.go:356] "Generic (PLEG): container finished" podID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerID="a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a" exitCode=0
Oct 30 18:38:14.985058 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.985028    4544 generic.go:356] "Generic (PLEG): container finished" podID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerID="cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3" exitCode=0
Oct 30 18:38:14.985058 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.985055    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0026cf83-5ea2-4ccd-a916-83763f467e6b","Type":"ContainerDied","Data":"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80"}
Oct 30 18:38:14.985387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.985064    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:14.985387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.985113    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0026cf83-5ea2-4ccd-a916-83763f467e6b","Type":"ContainerDied","Data":"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590"}
Oct 30 18:38:14.985387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.985143    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0026cf83-5ea2-4ccd-a916-83763f467e6b","Type":"ContainerDied","Data":"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de"}
Oct 30 18:38:14.985387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.985173    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0026cf83-5ea2-4ccd-a916-83763f467e6b","Type":"ContainerDied","Data":"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1"}
Oct 30 18:38:14.985387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.985191    4544 scope.go:122] "RemoveContainer" containerID="980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80"
Oct 30 18:38:14.985387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.985211    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0026cf83-5ea2-4ccd-a916-83763f467e6b","Type":"ContainerDied","Data":"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a"}
Oct 30 18:38:14.985387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.985248    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0026cf83-5ea2-4ccd-a916-83763f467e6b","Type":"ContainerDied","Data":"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3"}
Oct 30 18:38:14.985387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.985287    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0026cf83-5ea2-4ccd-a916-83763f467e6b","Type":"ContainerDied","Data":"cd9a84e38eecb1846e9f6124e298d4780c8ad35c2573655fd121466040b8c97c"}
Oct 30 18:38:14.993769 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:14.993754    4544 scope.go:122] "RemoveContainer" containerID="701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590"
Oct 30 18:38:15.000371 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.000353    4544 scope.go:122] "RemoveContainer" containerID="8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de"
Oct 30 18:38:15.006659 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.006640    4544 scope.go:122] "RemoveContainer" containerID="a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1"
Oct 30 18:38:15.013048 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.013030    4544 scope.go:122] "RemoveContainer" containerID="a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a"
Oct 30 18:38:15.018987 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.018971    4544 scope.go:122] "RemoveContainer" containerID="cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3"
Oct 30 18:38:15.025756 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.025740    4544 scope.go:122] "RemoveContainer" containerID="691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812"
Oct 30 18:38:15.032191 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.032173    4544 scope.go:122] "RemoveContainer" containerID="980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80"
Oct 30 18:38:15.032449 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:38:15.032428    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80\": container with ID starting with 980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80 not found: ID does not exist" containerID="980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80"
Oct 30 18:38:15.032493 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.032458    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80"} err="failed to get container status \"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80\": rpc error: code = NotFound desc = could not find container \"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80\": container with ID starting with 980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80 not found: ID does not exist"
Oct 30 18:38:15.032493 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.032489    4544 scope.go:122] "RemoveContainer" containerID="701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590"
Oct 30 18:38:15.032734 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:38:15.032715    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590\": container with ID starting with 701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590 not found: ID does not exist" containerID="701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590"
Oct 30 18:38:15.032796 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.032740    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590"} err="failed to get container status \"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590\": rpc error: code = NotFound desc = could not find container \"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590\": container with ID starting with 701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590 not found: ID does not exist"
Oct 30 18:38:15.032796 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.032770    4544 scope.go:122] "RemoveContainer" containerID="8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de"
Oct 30 18:38:15.032978 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:38:15.032962    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de\": container with ID starting with 8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de not found: ID does not exist" containerID="8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de"
Oct 30 18:38:15.033011 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.032983    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de"} err="failed to get container status \"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de\": rpc error: code = NotFound desc = could not find container \"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de\": container with ID starting with 8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de not found: ID does not exist"
Oct 30 18:38:15.033011 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.032995    4544 scope.go:122] "RemoveContainer" containerID="a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1"
Oct 30 18:38:15.033225 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:38:15.033208    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1\": container with ID starting with a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1 not found: ID does not exist" containerID="a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1"
Oct 30 18:38:15.033263 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.033228    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1"} err="failed to get container status \"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1\": rpc error: code = NotFound desc = could not find container \"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1\": container with ID starting with a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1 not found: ID does not exist"
Oct 30 18:38:15.033263 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.033240    4544 scope.go:122] "RemoveContainer" containerID="a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a"
Oct 30 18:38:15.033448 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:38:15.033433    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a\": container with ID starting with a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a not found: ID does not exist" containerID="a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a"
Oct 30 18:38:15.033487 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.033452    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a"} err="failed to get container status \"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a\": rpc error: code = NotFound desc = could not find container \"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a\": container with ID starting with a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a not found: ID does not exist"
Oct 30 18:38:15.033487 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.033462    4544 scope.go:122] "RemoveContainer" containerID="cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3"
Oct 30 18:38:15.033713 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:38:15.033690    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3\": container with ID starting with cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3 not found: ID does not exist" containerID="cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3"
Oct 30 18:38:15.033764 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.033718    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3"} err="failed to get container status \"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3\": rpc error: code = NotFound desc = could not find container \"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3\": container with ID starting with cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3 not found: ID does not exist"
Oct 30 18:38:15.033764 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.033731    4544 scope.go:122] "RemoveContainer" containerID="691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812"
Oct 30 18:38:15.033932 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:38:15.033915    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812\": container with ID starting with 691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812 not found: ID does not exist" containerID="691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812"
Oct 30 18:38:15.033932 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.033931    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812"} err="failed to get container status \"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812\": rpc error: code = NotFound desc = could not find container \"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812\": container with ID starting with 691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812 not found: ID does not exist"
Oct 30 18:38:15.034075 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.033941    4544 scope.go:122] "RemoveContainer" containerID="980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80"
Oct 30 18:38:15.034151 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.034135    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80"} err="failed to get container status \"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80\": rpc error: code = NotFound desc = could not find container \"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80\": container with ID starting with 980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80 not found: ID does not exist"
Oct 30 18:38:15.034194 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.034151    4544 scope.go:122] "RemoveContainer" containerID="701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590"
Oct 30 18:38:15.034345 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.034330    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590"} err="failed to get container status \"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590\": rpc error: code = NotFound desc = could not find container \"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590\": container with ID starting with 701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590 not found: ID does not exist"
Oct 30 18:38:15.034345 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.034344    4544 scope.go:122] "RemoveContainer" containerID="8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de"
Oct 30 18:38:15.034593 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.034576    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de"} err="failed to get container status \"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de\": rpc error: code = NotFound desc = could not find container \"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de\": container with ID starting with 8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de not found: ID does not exist"
Oct 30 18:38:15.034593 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.034592    4544 scope.go:122] "RemoveContainer" containerID="a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1"
Oct 30 18:38:15.034813 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.034793    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1"} err="failed to get container status \"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1\": rpc error: code = NotFound desc = could not find container \"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1\": container with ID starting with a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1 not found: ID does not exist"
Oct 30 18:38:15.034813 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.034812    4544 scope.go:122] "RemoveContainer" containerID="a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a"
Oct 30 18:38:15.035029 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.035009    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a"} err="failed to get container status \"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a\": rpc error: code = NotFound desc = could not find container \"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a\": container with ID starting with a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a not found: ID does not exist"
Oct 30 18:38:15.035097 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.035030    4544 scope.go:122] "RemoveContainer" containerID="cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3"
Oct 30 18:38:15.035264 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.035247    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3"} err="failed to get container status \"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3\": rpc error: code = NotFound desc = could not find container \"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3\": container with ID starting with cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3 not found: ID does not exist"
Oct 30 18:38:15.035326 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.035263    4544 scope.go:122] "RemoveContainer" containerID="691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812"
Oct 30 18:38:15.035480 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.035462    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812"} err="failed to get container status \"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812\": rpc error: code = NotFound desc = could not find container \"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812\": container with ID starting with 691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812 not found: ID does not exist"
Oct 30 18:38:15.035544 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.035482    4544 scope.go:122] "RemoveContainer" containerID="980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80"
Oct 30 18:38:15.035762 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.035732    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80"} err="failed to get container status \"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80\": rpc error: code = NotFound desc = could not find container \"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80\": container with ID starting with 980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80 not found: ID does not exist"
Oct 30 18:38:15.035762 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.035759    4544 scope.go:122] "RemoveContainer" containerID="701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590"
Oct 30 18:38:15.035976 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.035957    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590"} err="failed to get container status \"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590\": rpc error: code = NotFound desc = could not find container \"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590\": container with ID starting with 701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590 not found: ID does not exist"
Oct 30 18:38:15.036031 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.035976    4544 scope.go:122] "RemoveContainer" containerID="8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de"
Oct 30 18:38:15.036180 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.036159    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de"} err="failed to get container status \"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de\": rpc error: code = NotFound desc = could not find container \"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de\": container with ID starting with 8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de not found: ID does not exist"
Oct 30 18:38:15.036225 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.036179    4544 scope.go:122] "RemoveContainer" containerID="a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1"
Oct 30 18:38:15.036374 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.036356    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1"} err="failed to get container status \"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1\": rpc error: code = NotFound desc = could not find container \"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1\": container with ID starting with a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1 not found: ID does not exist"
Oct 30 18:38:15.036415 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.036374    4544 scope.go:122] "RemoveContainer" containerID="a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a"
Oct 30 18:38:15.036568 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.036551    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a"} err="failed to get container status \"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a\": rpc error: code = NotFound desc = could not find container \"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a\": container with ID starting with a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a not found: ID does not exist"
Oct 30 18:38:15.036568 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.036566    4544 scope.go:122] "RemoveContainer" containerID="cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3"
Oct 30 18:38:15.036802 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.036786    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3"} err="failed to get container status \"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3\": rpc error: code = NotFound desc = could not find container \"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3\": container with ID starting with cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3 not found: ID does not exist"
Oct 30 18:38:15.036802 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.036801    4544 scope.go:122] "RemoveContainer" containerID="691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812"
Oct 30 18:38:15.036974 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.036954    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812"} err="failed to get container status \"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812\": rpc error: code = NotFound desc = could not find container \"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812\": container with ID starting with 691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812 not found: ID does not exist"
Oct 30 18:38:15.037011 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.036975    4544 scope.go:122] "RemoveContainer" containerID="980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80"
Oct 30 18:38:15.037138 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.037120    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80"} err="failed to get container status \"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80\": rpc error: code = NotFound desc = could not find container \"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80\": container with ID starting with 980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80 not found: ID does not exist"
Oct 30 18:38:15.037192 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.037138    4544 scope.go:122] "RemoveContainer" containerID="701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590"
Oct 30 18:38:15.037374 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.037360    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590"} err="failed to get container status \"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590\": rpc error: code = NotFound desc = could not find container \"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590\": container with ID starting with 701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590 not found: ID does not exist"
Oct 30 18:38:15.037421 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.037374    4544 scope.go:122] "RemoveContainer" containerID="8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de"
Oct 30 18:38:15.037527 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.037513    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de"} err="failed to get container status \"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de\": rpc error: code = NotFound desc = could not find container \"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de\": container with ID starting with 8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de not found: ID does not exist"
Oct 30 18:38:15.037581 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.037532    4544 scope.go:122] "RemoveContainer" containerID="a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1"
Oct 30 18:38:15.037714 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.037701    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1"} err="failed to get container status \"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1\": rpc error: code = NotFound desc = could not find container \"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1\": container with ID starting with a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1 not found: ID does not exist"
Oct 30 18:38:15.037762 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.037713    4544 scope.go:122] "RemoveContainer" containerID="a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a"
Oct 30 18:38:15.037908 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.037890    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a"} err="failed to get container status \"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a\": rpc error: code = NotFound desc = could not find container \"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a\": container with ID starting with a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a not found: ID does not exist"
Oct 30 18:38:15.037951 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.037908    4544 scope.go:122] "RemoveContainer" containerID="cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3"
Oct 30 18:38:15.038089 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.038075    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3"} err="failed to get container status \"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3\": rpc error: code = NotFound desc = could not find container \"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3\": container with ID starting with cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3 not found: ID does not exist"
Oct 30 18:38:15.038130 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.038089    4544 scope.go:122] "RemoveContainer" containerID="691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812"
Oct 30 18:38:15.038258 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.038241    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812"} err="failed to get container status \"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812\": rpc error: code = NotFound desc = could not find container \"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812\": container with ID starting with 691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812 not found: ID does not exist"
Oct 30 18:38:15.038258 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.038257    4544 scope.go:122] "RemoveContainer" containerID="980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80"
Oct 30 18:38:15.038489 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.038472    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80"} err="failed to get container status \"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80\": rpc error: code = NotFound desc = could not find container \"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80\": container with ID starting with 980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80 not found: ID does not exist"
Oct 30 18:38:15.038489 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.038489    4544 scope.go:122] "RemoveContainer" containerID="701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590"
Oct 30 18:38:15.038726 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.038707    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590"} err="failed to get container status \"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590\": rpc error: code = NotFound desc = could not find container \"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590\": container with ID starting with 701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590 not found: ID does not exist"
Oct 30 18:38:15.038726 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.038726    4544 scope.go:122] "RemoveContainer" containerID="8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de"
Oct 30 18:38:15.038919 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.038900    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de"} err="failed to get container status \"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de\": rpc error: code = NotFound desc = could not find container \"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de\": container with ID starting with 8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de not found: ID does not exist"
Oct 30 18:38:15.038958 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.038919    4544 scope.go:122] "RemoveContainer" containerID="a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1"
Oct 30 18:38:15.039133 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.039115    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1"} err="failed to get container status \"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1\": rpc error: code = NotFound desc = could not find container \"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1\": container with ID starting with a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1 not found: ID does not exist"
Oct 30 18:38:15.039173 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.039133    4544 scope.go:122] "RemoveContainer" containerID="a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a"
Oct 30 18:38:15.039333 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.039317    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a"} err="failed to get container status \"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a\": rpc error: code = NotFound desc = could not find container \"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a\": container with ID starting with a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a not found: ID does not exist"
Oct 30 18:38:15.039333 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.039332    4544 scope.go:122] "RemoveContainer" containerID="cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3"
Oct 30 18:38:15.039540 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.039522    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3"} err="failed to get container status \"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3\": rpc error: code = NotFound desc = could not find container \"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3\": container with ID starting with cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3 not found: ID does not exist"
Oct 30 18:38:15.039583 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.039541    4544 scope.go:122] "RemoveContainer" containerID="691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812"
Oct 30 18:38:15.039753 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.039736    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812"} err="failed to get container status \"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812\": rpc error: code = NotFound desc = could not find container \"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812\": container with ID starting with 691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812 not found: ID does not exist"
Oct 30 18:38:15.039753 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.039752    4544 scope.go:122] "RemoveContainer" containerID="980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80"
Oct 30 18:38:15.039955 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.039936    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80"} err="failed to get container status \"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80\": rpc error: code = NotFound desc = could not find container \"980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80\": container with ID starting with 980b9f09f20ead3a3abe1d1a135e8dcd82faca017b7660ebe2cf40a8bddbfd80 not found: ID does not exist"
Oct 30 18:38:15.039995 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.039954    4544 scope.go:122] "RemoveContainer" containerID="701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590"
Oct 30 18:38:15.040141 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.040126    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590"} err="failed to get container status \"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590\": rpc error: code = NotFound desc = could not find container \"701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590\": container with ID starting with 701f1109d0f7c0a1fe68e7c01e35c86f782f1e18e032e0dfc48c4bef396bc590 not found: ID does not exist"
Oct 30 18:38:15.040141 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.040139    4544 scope.go:122] "RemoveContainer" containerID="8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de"
Oct 30 18:38:15.040336 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.040319    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de"} err="failed to get container status \"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de\": rpc error: code = NotFound desc = could not find container \"8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de\": container with ID starting with 8d4bfe33098193016f88883acfcba34065c6429a2cd76ce9288dcb70e7a095de not found: ID does not exist"
Oct 30 18:38:15.040376 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.040335    4544 scope.go:122] "RemoveContainer" containerID="a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1"
Oct 30 18:38:15.040520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.040505    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1"} err="failed to get container status \"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1\": rpc error: code = NotFound desc = could not find container \"a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1\": container with ID starting with a41d5466655d45dadf88cd9ed94d3fb59e32469b9db1a629613e31460a1199f1 not found: ID does not exist"
Oct 30 18:38:15.040520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.040518    4544 scope.go:122] "RemoveContainer" containerID="a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a"
Oct 30 18:38:15.040725 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.040708    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a"} err="failed to get container status \"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a\": rpc error: code = NotFound desc = could not find container \"a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a\": container with ID starting with a74bd9aabc66c0753e045269a79b22c46881611a638512415b8953d5604c5b5a not found: ID does not exist"
Oct 30 18:38:15.040725 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.040724    4544 scope.go:122] "RemoveContainer" containerID="cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3"
Oct 30 18:38:15.040928 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.040913    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3"} err="failed to get container status \"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3\": rpc error: code = NotFound desc = could not find container \"cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3\": container with ID starting with cff37764b09cd3af203c16649d3022fda45c8bb100255f2f79f6d8714f260da3 not found: ID does not exist"
Oct 30 18:38:15.040968 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.040929    4544 scope.go:122] "RemoveContainer" containerID="691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812"
Oct 30 18:38:15.041130 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.041111    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812"} err="failed to get container status \"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812\": rpc error: code = NotFound desc = could not find container \"691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812\": container with ID starting with 691d5fa644a146f9022bc7e472d39ddf6e0e514ddcb52ff7aad56b03f66cd812 not found: ID does not exist"
Oct 30 18:38:15.144586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.144553    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:38:15.150619 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.150575    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:38:15.175203 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175161    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:38:15.175394 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175381    4544 cpu_manager.go:414] "containerMap: removing container" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="thanos-sidecar"
Oct 30 18:38:15.175394 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175393    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="thanos-sidecar"
Oct 30 18:38:15.175520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175403    4544 cpu_manager.go:414] "containerMap: removing container" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="kube-rbac-proxy-web"
Oct 30 18:38:15.175520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175408    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="kube-rbac-proxy-web"
Oct 30 18:38:15.175520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175414    4544 cpu_manager.go:414] "containerMap: removing container" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="kube-rbac-proxy-thanos"
Oct 30 18:38:15.175520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175420    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="kube-rbac-proxy-thanos"
Oct 30 18:38:15.175520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175432    4544 cpu_manager.go:414] "containerMap: removing container" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="prometheus"
Oct 30 18:38:15.175520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175437    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="prometheus"
Oct 30 18:38:15.175520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175443    4544 cpu_manager.go:414] "containerMap: removing container" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="kube-rbac-proxy"
Oct 30 18:38:15.175520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175448    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="kube-rbac-proxy"
Oct 30 18:38:15.175520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175456    4544 cpu_manager.go:414] "containerMap: removing container" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="config-reloader"
Oct 30 18:38:15.175520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175461    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="config-reloader"
Oct 30 18:38:15.175520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175466    4544 cpu_manager.go:414] "containerMap: removing container" podUID="a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3" containerName="insights-gathering"
Oct 30 18:38:15.175520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175470    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3" containerName="insights-gathering"
Oct 30 18:38:15.175520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175478    4544 cpu_manager.go:414] "containerMap: removing container" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="init-config-reloader"
Oct 30 18:38:15.175520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175482    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="init-config-reloader"
Oct 30 18:38:15.176212 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175547    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="prometheus"
Oct 30 18:38:15.176212 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175553    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="kube-rbac-proxy"
Oct 30 18:38:15.176212 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175559    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="config-reloader"
Oct 30 18:38:15.176212 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175564    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="a98b7c0d-2a11-4f58-ab3c-76e8bb2ae3c3" containerName="insights-gathering"
Oct 30 18:38:15.176212 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175571    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="thanos-sidecar"
Oct 30 18:38:15.176212 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175575    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="kube-rbac-proxy-web"
Oct 30 18:38:15.176212 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.175580    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" containerName="kube-rbac-proxy-thanos"
Oct 30 18:38:15.179123 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.179103    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.229216 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.229193    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 18:38:15.229340 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.229226    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 18:38:15.229340 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.229235    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 18:38:15.229530 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.229511    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rs5tw\""
Oct 30 18:38:15.229530 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.229514    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 18:38:15.229664 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.229531    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 18:38:15.229664 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.229525    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 18:38:15.229664 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.229546    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 18:38:15.229789 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.229730    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 18:38:15.229789 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.229731    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 18:38:15.229789 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.229757    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5ccr3ko96fg85\""
Oct 30 18:38:15.229986 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.229974    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 18:38:15.229986 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.229980    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 18:38:15.230058 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.230034    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 18:38:15.230553 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.230536    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 18:38:15.234895 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.234875    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235024 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.234905    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-web-config\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235024 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.234923    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235024 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.234945    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235024 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.234996    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a13bc23a-de14-4bdd-abae-2b4a146d08c2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235162 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.235089    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d286r\" (UniqueName: \"kubernetes.io/projected/a13bc23a-de14-4bdd-abae-2b4a146d08c2-kube-api-access-d286r\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235162 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.235124    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a13bc23a-de14-4bdd-abae-2b4a146d08c2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235227 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.235187    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235260 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.235231    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235260 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.235251    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235323 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.235268    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235323 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.235282    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235323 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.235310    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235432 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.235369    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-config\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235432 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.235387    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235432 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.235405    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235534 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.235428    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235534 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.235465    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235593 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.235538    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a13bc23a-de14-4bdd-abae-2b4a146d08c2-config-out\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.235593 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.235556    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.276338 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.276313    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 18:38:15.276662 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.276536    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 18:38:15.336745 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.336713    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.336745 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.336746    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.336957 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.336763    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.336957 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.336780    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.336957 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.336845    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-config\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.336957 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.336876    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.337167 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.337036    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.337167 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.337073    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.337167 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.337097    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.337167 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.337122    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a13bc23a-de14-4bdd-abae-2b4a146d08c2-config-out\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.337167 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.337138    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.337395 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.337179    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.337395 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.337208    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-web-config\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.337581 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.337548    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.337819 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.337795    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.337955 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.337854    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.338055 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.337879    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.338197 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.338173    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.338260 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.338181    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.338260 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.338242    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a13bc23a-de14-4bdd-abae-2b4a146d08c2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.338363 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.338268    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d286r\" (UniqueName: \"kubernetes.io/projected/a13bc23a-de14-4bdd-abae-2b4a146d08c2-kube-api-access-d286r\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.338363 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.338295    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a13bc23a-de14-4bdd-abae-2b4a146d08c2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.338363 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.338316    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.338363 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.338343    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.339245 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.339223    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.339400 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.337962    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.340075 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.340050    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-config\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.340179 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.340103    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-web-config\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.340274 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.340245    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.340635 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.340596    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.340712 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.340615    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.340875 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.340813    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a13bc23a-de14-4bdd-abae-2b4a146d08c2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.341105 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.341061    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.341362 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.341341    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.342321 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.342273    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.342515 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.342492    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a13bc23a-de14-4bdd-abae-2b4a146d08c2-config-out\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.343152 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.343132    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a13bc23a-de14-4bdd-abae-2b4a146d08c2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.343284 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.343194    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a13bc23a-de14-4bdd-abae-2b4a146d08c2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.343760 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.343743    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a13bc23a-de14-4bdd-abae-2b4a146d08c2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.393966 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.393947    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d286r\" (UniqueName: \"kubernetes.io/projected/a13bc23a-de14-4bdd-abae-2b4a146d08c2-kube-api-access-d286r\") pod \"prometheus-k8s-0\" (UID: \"a13bc23a-de14-4bdd-abae-2b4a146d08c2\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.416450 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.416422    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:38:15.495574 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.495513    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:15.911297 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.911264    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 18:38:15.915015 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:38:15.914975    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda13bc23a_de14_4bdd_abae_2b4a146d08c2.slice/crio-8bc55eb41516fb5590c1c4fe8f6b4d08a3cdfd7d0b76e984539d5f162e389cb5 WatchSource:0}: Error finding container 8bc55eb41516fb5590c1c4fe8f6b4d08a3cdfd7d0b76e984539d5f162e389cb5: Status 404 returned error can't find the container with id 8bc55eb41516fb5590c1c4fe8f6b4d08a3cdfd7d0b76e984539d5f162e389cb5
Oct 30 18:38:15.990504 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.990477    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a13bc23a-de14-4bdd-abae-2b4a146d08c2","Type":"ContainerStarted","Data":"50183b0a52294e50fed78dd0f1212164508186c9b70bfe9c0f4c159975b3e890"}
Oct 30 18:38:15.990597 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:15.990551    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a13bc23a-de14-4bdd-abae-2b4a146d08c2","Type":"ContainerStarted","Data":"8bc55eb41516fb5590c1c4fe8f6b4d08a3cdfd7d0b76e984539d5f162e389cb5"}
Oct 30 18:38:16.600411 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:16.600381    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0026cf83-5ea2-4ccd-a916-83763f467e6b" path="/var/lib/kubelet/pods/0026cf83-5ea2-4ccd-a916-83763f467e6b/volumes"
Oct 30 18:38:16.996621 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:16.996514    4544 generic.go:356] "Generic (PLEG): container finished" podID="a13bc23a-de14-4bdd-abae-2b4a146d08c2" containerID="50183b0a52294e50fed78dd0f1212164508186c9b70bfe9c0f4c159975b3e890" exitCode=0
Oct 30 18:38:16.996621 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:16.996559    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a13bc23a-de14-4bdd-abae-2b4a146d08c2","Type":"ContainerDied","Data":"50183b0a52294e50fed78dd0f1212164508186c9b70bfe9c0f4c159975b3e890"}
Oct 30 18:38:16.996787 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:16.996643    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a13bc23a-de14-4bdd-abae-2b4a146d08c2","Type":"ContainerStarted","Data":"039e6958593eef367a610cfc974bfd549977502946eebe2328b3cdebf3a5dd63"}
Oct 30 18:38:16.996787 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:16.996685    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a13bc23a-de14-4bdd-abae-2b4a146d08c2","Type":"ContainerStarted","Data":"40052a6df7ec3ae32f97da0563871d5b63da0c7e7c616a9abfa7d724ac81a772"}
Oct 30 18:38:16.996787 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:16.996729    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a13bc23a-de14-4bdd-abae-2b4a146d08c2","Type":"ContainerStarted","Data":"3252601b8952b632f603d9ecc465f559e90773f1fcff30dfe9e480b6c90afb59"}
Oct 30 18:38:16.996787 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:16.996766    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a13bc23a-de14-4bdd-abae-2b4a146d08c2","Type":"ContainerStarted","Data":"3b4130ffef52417b0cba1a4e9549e08cece10be63a4c9a87bfe9ac07812c8a2d"}
Oct 30 18:38:16.996901 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:16.996808    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a13bc23a-de14-4bdd-abae-2b4a146d08c2","Type":"ContainerStarted","Data":"21aadcc0dffd514ebe3ee7dc61c47f87c3cc9386fa0e575a2c02fb86910c5863"}
Oct 30 18:38:16.996901 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:16.996845    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a13bc23a-de14-4bdd-abae-2b4a146d08c2","Type":"ContainerStarted","Data":"dfd7899fd8702ec85d19926eeb9d7d9a3c36aa7e32ccf0637f61f473e4a84d7b"}
Oct 30 18:38:17.116520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:17.116468    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.101492391 podStartE2EDuration="2.11645513s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.962739ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:17.116201652 +0000 UTC m=+94.642076784" watchObservedRunningTime="2026-10-30 18:38:17.11645513 +0000 UTC m=+94.642330226"
Oct 30 18:38:20.495940 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:38:20.495897    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:38:35.529412 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:38:35.529377    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:39:05.530738 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:39:05.530658    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:39:15.496424 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:39:15.496379    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:39:15.511464 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:39:15.511435    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:39:16.215749 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:39:16.215717    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 18:39:35.529686 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:39:35.529650    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:40:05.530039 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:40:05.529992    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:40:35.530467 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:40:35.530431    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:41:05.530379 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:41:05.530342    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:41:15.531885 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.531856    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"]
Oct 30 18:41:15.535085 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.535069    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.641737 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.641705    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e61c8a9d-9926-46cb-85e7-b704618d5348-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs\" (UID: \"e61c8a9d-9926-46cb-85e7-b704618d5348\") " pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.641870 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.641746    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e61c8a9d-9926-46cb-85e7-b704618d5348-tmp\") pod \"opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs\" (UID: \"e61c8a9d-9926-46cb-85e7-b704618d5348\") " pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.641870 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.641822    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-972jk\" (UniqueName: \"kubernetes.io/projected/e61c8a9d-9926-46cb-85e7-b704618d5348-kube-api-access-972jk\") pod \"opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs\" (UID: \"e61c8a9d-9926-46cb-85e7-b704618d5348\") " pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.641870 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.641841    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e61c8a9d-9926-46cb-85e7-b704618d5348-metrics-cert\") pod \"opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs\" (UID: \"e61c8a9d-9926-46cb-85e7-b704618d5348\") " pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.641870 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.641860    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e61c8a9d-9926-46cb-85e7-b704618d5348-webhook-cert\") pod \"opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs\" (UID: \"e61c8a9d-9926-46cb-85e7-b704618d5348\") " pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.641994 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.641922    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e61c8a9d-9926-46cb-85e7-b704618d5348-manifests\") pod \"opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs\" (UID: \"e61c8a9d-9926-46cb-85e7-b704618d5348\") " pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.706267 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.706244    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 18:41:15.706479 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.706464    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-metrics-tls\""
Oct 30 18:41:15.706576 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.706561    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 18:41:15.706681 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.706659    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-t8d96\""
Oct 30 18:41:15.706733 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.706666    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 18:41:15.706733 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.706681    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 18:41:15.743186 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.743166    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e61c8a9d-9926-46cb-85e7-b704618d5348-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs\" (UID: \"e61c8a9d-9926-46cb-85e7-b704618d5348\") " pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.743266 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.743198    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e61c8a9d-9926-46cb-85e7-b704618d5348-tmp\") pod \"opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs\" (UID: \"e61c8a9d-9926-46cb-85e7-b704618d5348\") " pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.743266 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.743228    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-972jk\" (UniqueName: \"kubernetes.io/projected/e61c8a9d-9926-46cb-85e7-b704618d5348-kube-api-access-972jk\") pod \"opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs\" (UID: \"e61c8a9d-9926-46cb-85e7-b704618d5348\") " pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.743266 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.743245    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e61c8a9d-9926-46cb-85e7-b704618d5348-metrics-cert\") pod \"opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs\" (UID: \"e61c8a9d-9926-46cb-85e7-b704618d5348\") " pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.743266 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.743263    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e61c8a9d-9926-46cb-85e7-b704618d5348-webhook-cert\") pod \"opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs\" (UID: \"e61c8a9d-9926-46cb-85e7-b704618d5348\") " pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.743473 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.743284    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e61c8a9d-9926-46cb-85e7-b704618d5348-manifests\") pod \"opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs\" (UID: \"e61c8a9d-9926-46cb-85e7-b704618d5348\") " pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.743765 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.743734    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e61c8a9d-9926-46cb-85e7-b704618d5348-manifests\") pod \"opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs\" (UID: \"e61c8a9d-9926-46cb-85e7-b704618d5348\") " pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.745463 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.745445    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e61c8a9d-9926-46cb-85e7-b704618d5348-tmp\") pod \"opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs\" (UID: \"e61c8a9d-9926-46cb-85e7-b704618d5348\") " pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.745740 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.745723    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e61c8a9d-9926-46cb-85e7-b704618d5348-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs\" (UID: \"e61c8a9d-9926-46cb-85e7-b704618d5348\") " pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.745789 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.745767    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e61c8a9d-9926-46cb-85e7-b704618d5348-metrics-cert\") pod \"opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs\" (UID: \"e61c8a9d-9926-46cb-85e7-b704618d5348\") " pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.745789 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.745767    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e61c8a9d-9926-46cb-85e7-b704618d5348-webhook-cert\") pod \"opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs\" (UID: \"e61c8a9d-9926-46cb-85e7-b704618d5348\") " pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.751153 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.751131    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"]
Oct 30 18:41:15.801717 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.801658    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-972jk\" (UniqueName: \"kubernetes.io/projected/e61c8a9d-9926-46cb-85e7-b704618d5348-kube-api-access-972jk\") pod \"opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs\" (UID: \"e61c8a9d-9926-46cb-85e7-b704618d5348\") " pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:15.850331 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:15.850305    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:16.324712 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:16.324680    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"]
Oct 30 18:41:16.327539 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:41:16.327514    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode61c8a9d_9926_46cb_85e7_b704618d5348.slice/crio-570d4c13bdd93ed1fc266fb3acdfcebbe5346c19f6417c950370142b3c238a2c WatchSource:0}: Error finding container 570d4c13bdd93ed1fc266fb3acdfcebbe5346c19f6417c950370142b3c238a2c: Status 404 returned error can't find the container with id 570d4c13bdd93ed1fc266fb3acdfcebbe5346c19f6417c950370142b3c238a2c
Oct 30 18:41:16.599173 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:16.599100    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs" event={"ID":"e61c8a9d-9926-46cb-85e7-b704618d5348","Type":"ContainerStarted","Data":"570d4c13bdd93ed1fc266fb3acdfcebbe5346c19f6417c950370142b3c238a2c"}
Oct 30 18:41:20.610576 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:20.610537    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs" event={"ID":"e61c8a9d-9926-46cb-85e7-b704618d5348","Type":"ContainerStarted","Data":"d37eba92d47e299ed13d22a74c39214b44e2ecb1f8d3716a1df7eabdabffa8a0"}
Oct 30 18:41:21.615264 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:21.615230    4544 generic.go:356] "Generic (PLEG): container finished" podID="e61c8a9d-9926-46cb-85e7-b704618d5348" containerID="d37eba92d47e299ed13d22a74c39214b44e2ecb1f8d3716a1df7eabdabffa8a0" exitCode=0
Oct 30 18:41:21.615687 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:21.615298    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs" event={"ID":"e61c8a9d-9926-46cb-85e7-b704618d5348","Type":"ContainerDied","Data":"d37eba92d47e299ed13d22a74c39214b44e2ecb1f8d3716a1df7eabdabffa8a0"}
Oct 30 18:41:22.619708 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:22.619672    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs" event={"ID":"e61c8a9d-9926-46cb-85e7-b704618d5348","Type":"ContainerStarted","Data":"5b91a1f4fe4541c6119ed77b65498efae5bed36d74136b36bd79b129982d1fee"}
Oct 30 18:41:22.620106 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:22.619804    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:22.835784 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:22.835733    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs" podStartSLOduration=2.7612857379999998 podStartE2EDuration="7.835719215s" totalImagesPullingTime="4.596970541s" totalInitContainerRuntime="477.462936ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:22.789354628 +0000 UTC m=+280.315229722" watchObservedRunningTime="2026-10-30 18:41:22.835719215 +0000 UTC m=+280.361594310"
Oct 30 18:41:33.625530 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:33.625503    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs"
Oct 30 18:41:35.529584 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:41:35.529551    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:41:40.348412 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.348382    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"]
Oct 30 18:41:40.354374 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.354355    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.414585 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.414559    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 18:41:40.430097 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.430074    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g98nz\" (UniqueName: \"kubernetes.io/projected/eb9521d9-d640-4c43-a474-c216fb24695f-kube-api-access-g98nz\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.430193 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.430105    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb9521d9-d640-4c43-a474-c216fb24695f-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.430193 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.430126    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.430266 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.430189    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/eb9521d9-d640-4c43-a474-c216fb24695f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.430266 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.430235    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.430327 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.430297    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb9521d9-d640-4c43-a474-c216fb24695f-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.430359 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.430336    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.453820 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.453797    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 18:41:40.453950 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.453924    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ntm7n\""
Oct 30 18:41:40.453950 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.453850    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 18:41:40.454119 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.453891    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 18:41:40.454119 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.453904    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 18:41:40.454119 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.453839    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 18:41:40.455063 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.455045    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"]
Oct 30 18:41:40.530877 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.530853    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb9521d9-d640-4c43-a474-c216fb24695f-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.531037 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.530884    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.531037 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.530925    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g98nz\" (UniqueName: \"kubernetes.io/projected/eb9521d9-d640-4c43-a474-c216fb24695f-kube-api-access-g98nz\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.531037 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.530965    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb9521d9-d640-4c43-a474-c216fb24695f-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.531037 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.530989    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.531037 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.531013    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/eb9521d9-d640-4c43-a474-c216fb24695f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.531299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.531067    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.531855 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.531824    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/eb9521d9-d640-4c43-a474-c216fb24695f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.533178 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.533156    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.533178 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.533167    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb9521d9-d640-4c43-a474-c216fb24695f-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.533342 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.533325    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.533399 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.533338    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.585346 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.585314    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb9521d9-d640-4c43-a474-c216fb24695f-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.586086 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.586064    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g98nz\" (UniqueName: \"kubernetes.io/projected/eb9521d9-d640-4c43-a474-c216fb24695f-kube-api-access-g98nz\") pod \"istiod-openshift-gateway-7876bfcb74-74qck\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:40.663367 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:40.663309    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:41.076496 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:41.076448    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"]
Oct 30 18:41:41.078982 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:41:41.078958    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb9521d9_d640_4c43_a474_c216fb24695f.slice/crio-babe09d3ef2420ed41e9f75469c38cee52ee4f8f21cf0f9abbc24d79eef32af1 WatchSource:0}: Error finding container babe09d3ef2420ed41e9f75469c38cee52ee4f8f21cf0f9abbc24d79eef32af1: Status 404 returned error can't find the container with id babe09d3ef2420ed41e9f75469c38cee52ee4f8f21cf0f9abbc24d79eef32af1
Oct 30 18:41:41.680766 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:41.680735    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck" event={"ID":"eb9521d9-d640-4c43-a474-c216fb24695f","Type":"ContainerStarted","Data":"babe09d3ef2420ed41e9f75469c38cee52ee4f8f21cf0f9abbc24d79eef32af1"}
Oct 30 18:41:42.610294 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:42.610264    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 18:41:42.612317 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:42.612297    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 18:41:45.181334 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:45.181305    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:41:45.181557 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:45.181386    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:41:45.701145 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:45.701107    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck" event={"ID":"eb9521d9-d640-4c43-a474-c216fb24695f","Type":"ContainerStarted","Data":"97a39a29ba9a114d76a63412f4866fb900e2c84518a1706543b465181febc527"}
Oct 30 18:41:45.701334 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:45.701247    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:45.904010 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:45.903930    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck" podStartSLOduration=1.804192779 podStartE2EDuration="5.90391818s" totalImagesPullingTime="4.099725401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:45.856894166 +0000 UTC m=+303.382769259" watchObservedRunningTime="2026-10-30 18:41:45.90391818 +0000 UTC m=+303.429793276"
Oct 30 18:41:46.707148 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:46.707113    4544 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-74qck container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 18:41:46.707548 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:46.707167    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck" podUID="eb9521d9-d640-4c43-a474-c216fb24695f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:41:49.706196 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:49.706162    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:41:50.241803 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.241772    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"]
Oct 30 18:41:50.246144 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.246121    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"
Oct 30 18:41:50.298223 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.298196    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:41:50.298223 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.298212    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Oct 30 18:41:50.298397 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.298220    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Oct 30 18:41:50.298397 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.298237    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Oct 30 18:41:50.298397 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.298292    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-whgwz\""
Oct 30 18:41:50.298397 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.298296    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Oct 30 18:41:50.298973 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.298949    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"]
Oct 30 18:41:50.410578 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.410547    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c966d54b-2ead-4d91-96ce-3880018bc913-cert\") pod \"lws-controller-manager-7b9b666444-v9rmw\" (UID: \"c966d54b-2ead-4d91-96ce-3880018bc913\") " pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"
Oct 30 18:41:50.410785 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.410616    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b599g\" (UniqueName: \"kubernetes.io/projected/c966d54b-2ead-4d91-96ce-3880018bc913-kube-api-access-b599g\") pod \"lws-controller-manager-7b9b666444-v9rmw\" (UID: \"c966d54b-2ead-4d91-96ce-3880018bc913\") " pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"
Oct 30 18:41:50.410785 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.410737    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c966d54b-2ead-4d91-96ce-3880018bc913-manager-config\") pod \"lws-controller-manager-7b9b666444-v9rmw\" (UID: \"c966d54b-2ead-4d91-96ce-3880018bc913\") " pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"
Oct 30 18:41:50.410908 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.410801    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c966d54b-2ead-4d91-96ce-3880018bc913-metrics-cert\") pod \"lws-controller-manager-7b9b666444-v9rmw\" (UID: \"c966d54b-2ead-4d91-96ce-3880018bc913\") " pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"
Oct 30 18:41:50.511258 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.511195    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b599g\" (UniqueName: \"kubernetes.io/projected/c966d54b-2ead-4d91-96ce-3880018bc913-kube-api-access-b599g\") pod \"lws-controller-manager-7b9b666444-v9rmw\" (UID: \"c966d54b-2ead-4d91-96ce-3880018bc913\") " pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"
Oct 30 18:41:50.511258 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.511238    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c966d54b-2ead-4d91-96ce-3880018bc913-manager-config\") pod \"lws-controller-manager-7b9b666444-v9rmw\" (UID: \"c966d54b-2ead-4d91-96ce-3880018bc913\") " pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"
Oct 30 18:41:50.511408 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.511260    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c966d54b-2ead-4d91-96ce-3880018bc913-metrics-cert\") pod \"lws-controller-manager-7b9b666444-v9rmw\" (UID: \"c966d54b-2ead-4d91-96ce-3880018bc913\") " pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"
Oct 30 18:41:50.511408 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.511296    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c966d54b-2ead-4d91-96ce-3880018bc913-cert\") pod \"lws-controller-manager-7b9b666444-v9rmw\" (UID: \"c966d54b-2ead-4d91-96ce-3880018bc913\") " pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"
Oct 30 18:41:50.511905 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.511882    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c966d54b-2ead-4d91-96ce-3880018bc913-manager-config\") pod \"lws-controller-manager-7b9b666444-v9rmw\" (UID: \"c966d54b-2ead-4d91-96ce-3880018bc913\") " pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"
Oct 30 18:41:50.513480 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.513457    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c966d54b-2ead-4d91-96ce-3880018bc913-cert\") pod \"lws-controller-manager-7b9b666444-v9rmw\" (UID: \"c966d54b-2ead-4d91-96ce-3880018bc913\") " pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"
Oct 30 18:41:50.513621 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.513584    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c966d54b-2ead-4d91-96ce-3880018bc913-metrics-cert\") pod \"lws-controller-manager-7b9b666444-v9rmw\" (UID: \"c966d54b-2ead-4d91-96ce-3880018bc913\") " pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"
Oct 30 18:41:50.564854 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.564829    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b599g\" (UniqueName: \"kubernetes.io/projected/c966d54b-2ead-4d91-96ce-3880018bc913-kube-api-access-b599g\") pod \"lws-controller-manager-7b9b666444-v9rmw\" (UID: \"c966d54b-2ead-4d91-96ce-3880018bc913\") " pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"
Oct 30 18:41:50.856958 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:50.856921    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"
Oct 30 18:41:51.221864 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:51.221801    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"]
Oct 30 18:41:51.225332 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:41:51.225308    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc966d54b_2ead_4d91_96ce_3880018bc913.slice/crio-9711245f3822bca98b25683bc09afdbb8ad90a21f35cd9e6a494773ec70d8e01 WatchSource:0}: Error finding container 9711245f3822bca98b25683bc09afdbb8ad90a21f35cd9e6a494773ec70d8e01: Status 404 returned error can't find the container with id 9711245f3822bca98b25683bc09afdbb8ad90a21f35cd9e6a494773ec70d8e01
Oct 30 18:41:51.227081 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:51.227063    4544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:41:51.722546 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:51.722510    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw" event={"ID":"c966d54b-2ead-4d91-96ce-3880018bc913","Type":"ContainerStarted","Data":"9711245f3822bca98b25683bc09afdbb8ad90a21f35cd9e6a494773ec70d8e01"}
Oct 30 18:41:55.738363 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:55.738282    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw" event={"ID":"c966d54b-2ead-4d91-96ce-3880018bc913","Type":"ContainerStarted","Data":"117f523905aa76f815e53028b1673cf3237f5aab6e189a22e316add70288fdbe"}
Oct 30 18:41:55.738363 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:55.738347    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"
Oct 30 18:41:55.846264 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:41:55.846214    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw" podStartSLOduration=1.648509072 podStartE2EDuration="5.846197148s" totalImagesPullingTime="4.197688076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:55.845192795 +0000 UTC m=+313.371067891" watchObservedRunningTime="2026-10-30 18:41:55.846197148 +0000 UTC m=+313.372072268"
Oct 30 18:42:05.531038 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:42:05.530999    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:42:06.743589 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:42:06.743563    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7b9b666444-v9rmw"
Oct 30 18:42:35.529727 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:42:35.529683    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:42:42.591837 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:42:42.591807    4544 scope.go:122] "RemoveContainer" containerID="f4ce3e1ab0b44b64afcdeb40f40b6839ebbd4563df941c82da0a4e0a4380e6b1"
Oct 30 18:43:05.529691 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:43:05.529649    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:43:07.892849 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:07.892816    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"]
Oct 30 18:43:07.893237 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:07.892993    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck" podUID="eb9521d9-d640-4c43-a474-c216fb24695f" containerName="discovery" containerID="cri-o://97a39a29ba9a114d76a63412f4866fb900e2c84518a1706543b465181febc527" gracePeriod=30
Oct 30 18:43:08.125455 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.125431    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:43:08.207317 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.207255    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb9521d9-d640-4c43-a474-c216fb24695f-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb9521d9-d640-4c43-a474-c216fb24695f-local-certs\") pod \"eb9521d9-d640-4c43-a474-c216fb24695f\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") "
Oct 30 18:43:08.207317 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.207292    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb9521d9-d640-4c43-a474-c216fb24695f-kube-api-access-g98nz\" (UniqueName: \"kubernetes.io/projected/eb9521d9-d640-4c43-a474-c216fb24695f-kube-api-access-g98nz\") pod \"eb9521d9-d640-4c43-a474-c216fb24695f\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") "
Oct 30 18:43:08.207519 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.207336    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb9521d9-d640-4c43-a474-c216fb24695f-istio-token\" (UniqueName: \"kubernetes.io/projected/eb9521d9-d640-4c43-a474-c216fb24695f-istio-token\") pod \"eb9521d9-d640-4c43-a474-c216fb24695f\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") "
Oct 30 18:43:08.207519 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.207361    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-istio-kubeconfig\") pod \"eb9521d9-d640-4c43-a474-c216fb24695f\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") "
Oct 30 18:43:08.207519 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.207417    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb9521d9-d640-4c43-a474-c216fb24695f-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/eb9521d9-d640-4c43-a474-c216fb24695f-istio-csr-ca-configmap\") pod \"eb9521d9-d640-4c43-a474-c216fb24695f\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") "
Oct 30 18:43:08.207519 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.207454    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-istio-csr-dns-cert\") pod \"eb9521d9-d640-4c43-a474-c216fb24695f\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") "
Oct 30 18:43:08.207519 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.207511    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-cacerts\" (UniqueName: \"kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-cacerts\") pod \"eb9521d9-d640-4c43-a474-c216fb24695f\" (UID: \"eb9521d9-d640-4c43-a474-c216fb24695f\") "
Oct 30 18:43:08.207932 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.207885    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb9521d9-d640-4c43-a474-c216fb24695f-istio-csr-ca-configmap" pod "eb9521d9-d640-4c43-a474-c216fb24695f" (UID: "eb9521d9-d640-4c43-a474-c216fb24695f"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:43:08.209741 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.209716    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb9521d9-d640-4c43-a474-c216fb24695f-istio-token" pod "eb9521d9-d640-4c43-a474-c216fb24695f" (UID: "eb9521d9-d640-4c43-a474-c216fb24695f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:43:08.209849 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.209731    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb9521d9-d640-4c43-a474-c216fb24695f-kube-api-access-g98nz" pod "eb9521d9-d640-4c43-a474-c216fb24695f" (UID: "eb9521d9-d640-4c43-a474-c216fb24695f"). InnerVolumeSpecName "kube-api-access-g98nz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:43:08.209892 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.209847    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb9521d9-d640-4c43-a474-c216fb24695f-local-certs" pod "eb9521d9-d640-4c43-a474-c216fb24695f" (UID: "eb9521d9-d640-4c43-a474-c216fb24695f"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:43:08.209996 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.209973    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-istio-csr-dns-cert" pod "eb9521d9-d640-4c43-a474-c216fb24695f" (UID: "eb9521d9-d640-4c43-a474-c216fb24695f"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:43:08.210048 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.210002    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-istio-kubeconfig" pod "eb9521d9-d640-4c43-a474-c216fb24695f" (UID: "eb9521d9-d640-4c43-a474-c216fb24695f"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:43:08.210084 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.210066    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-cacerts" pod "eb9521d9-d640-4c43-a474-c216fb24695f" (UID: "eb9521d9-d640-4c43-a474-c216fb24695f"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:43:08.308962 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.308935    4544 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/eb9521d9-d640-4c43-a474-c216fb24695f-istio-csr-ca-configmap\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:43:08.308962 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.308959    4544 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-istio-csr-dns-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:43:08.309123 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.308969    4544 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-cacerts\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:43:08.309123 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.308984    4544 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb9521d9-d640-4c43-a474-c216fb24695f-local-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:43:08.309123 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.308994    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g98nz\" (UniqueName: \"kubernetes.io/projected/eb9521d9-d640-4c43-a474-c216fb24695f-kube-api-access-g98nz\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:43:08.309123 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.309003    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb9521d9-d640-4c43-a474-c216fb24695f-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:43:08.309123 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.309011    4544 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eb9521d9-d640-4c43-a474-c216fb24695f-istio-kubeconfig\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:43:08.977102 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.977024    4544 generic.go:356] "Generic (PLEG): container finished" podID="eb9521d9-d640-4c43-a474-c216fb24695f" containerID="97a39a29ba9a114d76a63412f4866fb900e2c84518a1706543b465181febc527" exitCode=0
Oct 30 18:43:08.977102 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.977081    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"
Oct 30 18:43:08.977554 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.977103    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck" event={"ID":"eb9521d9-d640-4c43-a474-c216fb24695f","Type":"ContainerDied","Data":"97a39a29ba9a114d76a63412f4866fb900e2c84518a1706543b465181febc527"}
Oct 30 18:43:08.977554 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.977165    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck" event={"ID":"eb9521d9-d640-4c43-a474-c216fb24695f","Type":"ContainerDied","Data":"babe09d3ef2420ed41e9f75469c38cee52ee4f8f21cf0f9abbc24d79eef32af1"}
Oct 30 18:43:08.977554 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.977184    4544 scope.go:122] "RemoveContainer" containerID="97a39a29ba9a114d76a63412f4866fb900e2c84518a1706543b465181febc527"
Oct 30 18:43:08.984879 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.984861    4544 scope.go:122] "RemoveContainer" containerID="97a39a29ba9a114d76a63412f4866fb900e2c84518a1706543b465181febc527"
Oct 30 18:43:08.985139 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:43:08.985121    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97a39a29ba9a114d76a63412f4866fb900e2c84518a1706543b465181febc527\": container with ID starting with 97a39a29ba9a114d76a63412f4866fb900e2c84518a1706543b465181febc527 not found: ID does not exist" containerID="97a39a29ba9a114d76a63412f4866fb900e2c84518a1706543b465181febc527"
Oct 30 18:43:08.985186 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:08.985147    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a39a29ba9a114d76a63412f4866fb900e2c84518a1706543b465181febc527"} err="failed to get container status \"97a39a29ba9a114d76a63412f4866fb900e2c84518a1706543b465181febc527\": rpc error: code = NotFound desc = could not find container \"97a39a29ba9a114d76a63412f4866fb900e2c84518a1706543b465181febc527\": container with ID starting with 97a39a29ba9a114d76a63412f4866fb900e2c84518a1706543b465181febc527 not found: ID does not exist"
Oct 30 18:43:09.176805 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:09.176778    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"]
Oct 30 18:43:09.222857 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:09.222832    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-74qck"]
Oct 30 18:43:10.599973 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:43:10.599939    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb9521d9-d640-4c43-a474-c216fb24695f" path="/var/lib/kubelet/pods/eb9521d9-d640-4c43-a474-c216fb24695f/volumes"
Oct 30 18:43:35.530571 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:43:35.530533    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:44:05.530294 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:44:05.530254    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:44:31.041817 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:31.041784    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"]
Oct 30 18:44:31.042283 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:31.042067    4544 cpu_manager.go:414] "containerMap: removing container" podUID="eb9521d9-d640-4c43-a474-c216fb24695f" containerName="discovery"
Oct 30 18:44:31.042283 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:31.042081    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb9521d9-d640-4c43-a474-c216fb24695f" containerName="discovery"
Oct 30 18:44:31.042283 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:31.042121    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb9521d9-d640-4c43-a474-c216fb24695f" containerName="discovery"
Oct 30 18:44:31.044897 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:31.044882    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"
Oct 30 18:44:31.087740 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:31.087715    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"]
Oct 30 18:44:31.132777 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:31.132745    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 18:44:31.132900 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:31.132823    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 18:44:31.132900 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:31.132877    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wrjw5\""
Oct 30 18:44:31.200024 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:31.200000    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe49dc86-2dd1-410d-a21d-1c3033ab3ae9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-847vz\" (UID: \"fe49dc86-2dd1-410d-a21d-1c3033ab3ae9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"
Oct 30 18:44:31.200174 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:31.200035    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns9dn\" (UniqueName: \"kubernetes.io/projected/fe49dc86-2dd1-410d-a21d-1c3033ab3ae9-kube-api-access-ns9dn\") pod \"kuadrant-operator-controller-manager-6b6fb56846-847vz\" (UID: \"fe49dc86-2dd1-410d-a21d-1c3033ab3ae9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"
Oct 30 18:44:31.300832 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:31.300765    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe49dc86-2dd1-410d-a21d-1c3033ab3ae9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-847vz\" (UID: \"fe49dc86-2dd1-410d-a21d-1c3033ab3ae9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"
Oct 30 18:44:31.300832 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:31.300802    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns9dn\" (UniqueName: \"kubernetes.io/projected/fe49dc86-2dd1-410d-a21d-1c3033ab3ae9-kube-api-access-ns9dn\") pod \"kuadrant-operator-controller-manager-6b6fb56846-847vz\" (UID: \"fe49dc86-2dd1-410d-a21d-1c3033ab3ae9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"
Oct 30 18:44:31.301130 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:31.301111    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe49dc86-2dd1-410d-a21d-1c3033ab3ae9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-847vz\" (UID: \"fe49dc86-2dd1-410d-a21d-1c3033ab3ae9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"
Oct 30 18:44:31.362119 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:31.362089    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns9dn\" (UniqueName: \"kubernetes.io/projected/fe49dc86-2dd1-410d-a21d-1c3033ab3ae9-kube-api-access-ns9dn\") pod \"kuadrant-operator-controller-manager-6b6fb56846-847vz\" (UID: \"fe49dc86-2dd1-410d-a21d-1c3033ab3ae9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"
Oct 30 18:44:31.656358 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:31.656278    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"
Oct 30 18:44:32.131521 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:32.131470    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"]
Oct 30 18:44:32.136482 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:44:32.136451    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe49dc86_2dd1_410d_a21d_1c3033ab3ae9.slice/crio-88201d3ee2c4ad6444e4f65fb51ba686754c4da0d0c2f634578d57a68068a8bb WatchSource:0}: Error finding container 88201d3ee2c4ad6444e4f65fb51ba686754c4da0d0c2f634578d57a68068a8bb: Status 404 returned error can't find the container with id 88201d3ee2c4ad6444e4f65fb51ba686754c4da0d0c2f634578d57a68068a8bb
Oct 30 18:44:32.265245 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:32.265192    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz" event={"ID":"fe49dc86-2dd1-410d-a21d-1c3033ab3ae9","Type":"ContainerStarted","Data":"88201d3ee2c4ad6444e4f65fb51ba686754c4da0d0c2f634578d57a68068a8bb"}
Oct 30 18:44:33.378878 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:33.378847    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d6dd76dc-q9wt9"]
Oct 30 18:44:33.383090 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:33.383061    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d6dd76dc-q9wt9"
Oct 30 18:44:33.438685 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:33.438664    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-qpqjm\""
Oct 30 18:44:33.486533 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:33.486505    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d6dd76dc-q9wt9"]
Oct 30 18:44:33.516731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:33.516689    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95fk9\" (UniqueName: \"kubernetes.io/projected/59f3ebf1-4b49-4789-bb7c-cca228cfc72a-kube-api-access-95fk9\") pod \"authorino-operator-68d6dd76dc-q9wt9\" (UID: \"59f3ebf1-4b49-4789-bb7c-cca228cfc72a\") " pod="kuadrant-system/authorino-operator-68d6dd76dc-q9wt9"
Oct 30 18:44:33.617711 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:33.617681    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95fk9\" (UniqueName: \"kubernetes.io/projected/59f3ebf1-4b49-4789-bb7c-cca228cfc72a-kube-api-access-95fk9\") pod \"authorino-operator-68d6dd76dc-q9wt9\" (UID: \"59f3ebf1-4b49-4789-bb7c-cca228cfc72a\") " pod="kuadrant-system/authorino-operator-68d6dd76dc-q9wt9"
Oct 30 18:44:33.672831 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:33.672786    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95fk9\" (UniqueName: \"kubernetes.io/projected/59f3ebf1-4b49-4789-bb7c-cca228cfc72a-kube-api-access-95fk9\") pod \"authorino-operator-68d6dd76dc-q9wt9\" (UID: \"59f3ebf1-4b49-4789-bb7c-cca228cfc72a\") " pod="kuadrant-system/authorino-operator-68d6dd76dc-q9wt9"
Oct 30 18:44:33.695623 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:33.695583    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d6dd76dc-q9wt9"
Oct 30 18:44:34.070192 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:34.070165    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d6dd76dc-q9wt9"]
Oct 30 18:44:34.073330 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:44:34.073302    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59f3ebf1_4b49_4789_bb7c_cca228cfc72a.slice/crio-3c695c923e7c28260e518dea75617a49ad80feebbc4ec313e03bed29df914b70 WatchSource:0}: Error finding container 3c695c923e7c28260e518dea75617a49ad80feebbc4ec313e03bed29df914b70: Status 404 returned error can't find the container with id 3c695c923e7c28260e518dea75617a49ad80feebbc4ec313e03bed29df914b70
Oct 30 18:44:34.273527 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:34.273493    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d6dd76dc-q9wt9" event={"ID":"59f3ebf1-4b49-4789-bb7c-cca228cfc72a","Type":"ContainerStarted","Data":"3c695c923e7c28260e518dea75617a49ad80feebbc4ec313e03bed29df914b70"}
Oct 30 18:44:35.555148 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:44:35.555111    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:44:35.596031 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:35.595998    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq"]
Oct 30 18:44:35.604986 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:35.604962    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq"
Oct 30 18:44:35.631352 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:35.631321    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq5gb\" (UniqueName: \"kubernetes.io/projected/005122d5-6fc8-407a-aaee-80a3e6d0d4a0-kube-api-access-hq5gb\") pod \"limitador-operator-controller-manager-856dd584c5-m9qzq\" (UID: \"005122d5-6fc8-407a-aaee-80a3e6d0d4a0\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq"
Oct 30 18:44:35.653425 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:35.653390    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq"]
Oct 30 18:44:35.697234 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:35.697206    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-jtjdl\""
Oct 30 18:44:35.731994 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:35.731960    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq5gb\" (UniqueName: \"kubernetes.io/projected/005122d5-6fc8-407a-aaee-80a3e6d0d4a0-kube-api-access-hq5gb\") pod \"limitador-operator-controller-manager-856dd584c5-m9qzq\" (UID: \"005122d5-6fc8-407a-aaee-80a3e6d0d4a0\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq"
Oct 30 18:44:35.790088 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:35.790032    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq5gb\" (UniqueName: \"kubernetes.io/projected/005122d5-6fc8-407a-aaee-80a3e6d0d4a0-kube-api-access-hq5gb\") pod \"limitador-operator-controller-manager-856dd584c5-m9qzq\" (UID: \"005122d5-6fc8-407a-aaee-80a3e6d0d4a0\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq"
Oct 30 18:44:35.923744 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:35.923668    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq"
Oct 30 18:44:36.298324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:36.298290    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq"]
Oct 30 18:44:36.302896 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:44:36.302864    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod005122d5_6fc8_407a_aaee_80a3e6d0d4a0.slice/crio-7e8b15d69b9147b7ad1a7d535b4e3774629fe2c857478bccd68555773c1a9737 WatchSource:0}: Error finding container 7e8b15d69b9147b7ad1a7d535b4e3774629fe2c857478bccd68555773c1a9737: Status 404 returned error can't find the container with id 7e8b15d69b9147b7ad1a7d535b4e3774629fe2c857478bccd68555773c1a9737
Oct 30 18:44:37.286344 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:37.286309    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq" event={"ID":"005122d5-6fc8-407a-aaee-80a3e6d0d4a0","Type":"ContainerStarted","Data":"7e8b15d69b9147b7ad1a7d535b4e3774629fe2c857478bccd68555773c1a9737"}
Oct 30 18:44:37.287931 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:37.287903    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d6dd76dc-q9wt9" event={"ID":"59f3ebf1-4b49-4789-bb7c-cca228cfc72a","Type":"ContainerStarted","Data":"74f886ae1523ee9baed35728c2b211baf10712d4b04b2b3fe4799b07c1c13c5b"}
Oct 30 18:44:37.288172 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:37.288151    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d6dd76dc-q9wt9"
Oct 30 18:44:37.396461 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:37.396403    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d6dd76dc-q9wt9" podStartSLOduration=1.861119655 podStartE2EDuration="4.39638517s" totalImagesPullingTime="2.535265515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:37.395538021 +0000 UTC m=+474.921413120" watchObservedRunningTime="2026-10-30 18:44:37.39638517 +0000 UTC m=+474.922260270"
Oct 30 18:44:39.297678 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:39.297645    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq" event={"ID":"005122d5-6fc8-407a-aaee-80a3e6d0d4a0","Type":"ContainerStarted","Data":"3a9dbe1dd7bf743e566ba60a64fc0f9402dd1fd70ea0ac10f7e519772805b3c9"}
Oct 30 18:44:39.298096 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:39.297733    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq"
Oct 30 18:44:39.413701 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:39.413655    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq" podStartSLOduration=1.7665832780000001 podStartE2EDuration="4.413640371s" totalImagesPullingTime="2.647057093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:39.412884322 +0000 UTC m=+476.938759419" watchObservedRunningTime="2026-10-30 18:44:39.413640371 +0000 UTC m=+476.939515467"
Oct 30 18:44:40.303234 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:40.303187    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz" event={"ID":"fe49dc86-2dd1-410d-a21d-1c3033ab3ae9","Type":"ContainerStarted","Data":"40d1b5cd9a9e960a108f43fd272b53907775921da1d8e01e06253f8b223590a8"}
Oct 30 18:44:40.303742 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:40.303424    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"
Oct 30 18:44:40.505563 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:40.505215    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz" podStartSLOduration=3.28595424 podStartE2EDuration="10.505198027s" totalImagesPullingTime="7.219243787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:40.458648576 +0000 UTC m=+477.984523671" watchObservedRunningTime="2026-10-30 18:44:40.505198027 +0000 UTC m=+478.031073122"
Oct 30 18:44:48.295058 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:48.295029    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d6dd76dc-q9wt9"
Oct 30 18:44:50.305993 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:50.305964    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq"
Oct 30 18:44:51.309121 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:51.309095    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"
Oct 30 18:44:53.200676 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:53.200586    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm"]
Oct 30 18:44:53.204228 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:53.204210    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm"
Oct 30 18:44:53.247828 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:53.247799    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm"]
Oct 30 18:44:53.259016 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:53.258981    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a29a9f5-5ad9-4502-8090-576b2618c335-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b76784d48-8rhnm\" (UID: \"3a29a9f5-5ad9-4502-8090-576b2618c335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm"
Oct 30 18:44:53.259128 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:53.259108    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztkpv\" (UniqueName: \"kubernetes.io/projected/3a29a9f5-5ad9-4502-8090-576b2618c335-kube-api-access-ztkpv\") pod \"kuadrant-operator-controller-manager-6b76784d48-8rhnm\" (UID: \"3a29a9f5-5ad9-4502-8090-576b2618c335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm"
Oct 30 18:44:53.360383 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:53.360355    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztkpv\" (UniqueName: \"kubernetes.io/projected/3a29a9f5-5ad9-4502-8090-576b2618c335-kube-api-access-ztkpv\") pod \"kuadrant-operator-controller-manager-6b76784d48-8rhnm\" (UID: \"3a29a9f5-5ad9-4502-8090-576b2618c335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm"
Oct 30 18:44:53.360487 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:53.360392    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a29a9f5-5ad9-4502-8090-576b2618c335-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b76784d48-8rhnm\" (UID: \"3a29a9f5-5ad9-4502-8090-576b2618c335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm"
Oct 30 18:44:53.360817 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:53.360798    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a29a9f5-5ad9-4502-8090-576b2618c335-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b76784d48-8rhnm\" (UID: \"3a29a9f5-5ad9-4502-8090-576b2618c335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm"
Oct 30 18:44:53.424269 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:53.424241    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztkpv\" (UniqueName: \"kubernetes.io/projected/3a29a9f5-5ad9-4502-8090-576b2618c335-kube-api-access-ztkpv\") pod \"kuadrant-operator-controller-manager-6b76784d48-8rhnm\" (UID: \"3a29a9f5-5ad9-4502-8090-576b2618c335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm"
Oct 30 18:44:53.516361 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:53.516339    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm"
Oct 30 18:44:53.925381 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:53.925308    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm"]
Oct 30 18:44:53.928596 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:44:53.928573    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a29a9f5_5ad9_4502_8090_576b2618c335.slice/crio-d2fed6ebfcccab36eb50fb7f7a9c47f1114b7d0a2b25a45c4dedd7ea1b67225d WatchSource:0}: Error finding container d2fed6ebfcccab36eb50fb7f7a9c47f1114b7d0a2b25a45c4dedd7ea1b67225d: Status 404 returned error can't find the container with id d2fed6ebfcccab36eb50fb7f7a9c47f1114b7d0a2b25a45c4dedd7ea1b67225d
Oct 30 18:44:54.356012 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:54.355979    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm" event={"ID":"3a29a9f5-5ad9-4502-8090-576b2618c335","Type":"ContainerStarted","Data":"7b76c0ceb4a9652a7cdc40a5075c5b532528d6a16e277beb0350be107159dd8e"}
Oct 30 18:44:54.356387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:54.356033    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm" event={"ID":"3a29a9f5-5ad9-4502-8090-576b2618c335","Type":"ContainerStarted","Data":"d2fed6ebfcccab36eb50fb7f7a9c47f1114b7d0a2b25a45c4dedd7ea1b67225d"}
Oct 30 18:44:54.356387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:54.356068    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm"
Oct 30 18:44:54.518360 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:54.518303    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm" podStartSLOduration=1.5182891330000001 podStartE2EDuration="1.518289133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:54.516889991 +0000 UTC m=+492.042765086" watchObservedRunningTime="2026-10-30 18:44:54.518289133 +0000 UTC m=+492.044164221"
Oct 30 18:44:55.330623 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.330573    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"]
Oct 30 18:44:55.330797 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.330776    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz" podUID="fe49dc86-2dd1-410d-a21d-1c3033ab3ae9" containerName="manager" containerID="cri-o://40d1b5cd9a9e960a108f43fd272b53907775921da1d8e01e06253f8b223590a8" gracePeriod=2
Oct 30 18:44:55.377843 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.377810    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"]
Oct 30 18:44:55.384762 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.384715    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx"]
Oct 30 18:44:55.385390 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.385368    4544 cpu_manager.go:414] "containerMap: removing container" podUID="fe49dc86-2dd1-410d-a21d-1c3033ab3ae9" containerName="manager"
Oct 30 18:44:55.385492 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.385391    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe49dc86-2dd1-410d-a21d-1c3033ab3ae9" containerName="manager"
Oct 30 18:44:55.385571 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.385558    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe49dc86-2dd1-410d-a21d-1c3033ab3ae9" containerName="manager"
Oct 30 18:44:55.389000 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.388978    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx"
Oct 30 18:44:55.422420 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:44:55.422387    4544 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-847vz\" is forbidden: User \"system:node:ip-10-0-140-122.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-122.us-west-2.compute.internal' and this object" podUID="fe49dc86-2dd1-410d-a21d-1c3033ab3ae9" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"
Oct 30 18:44:55.423855 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.423827    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm"]
Oct 30 18:44:55.425684 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.425658    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm"]
Oct 30 18:44:55.425788 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.425716    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72"]
Oct 30 18:44:55.426024 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.426007    4544 cpu_manager.go:414] "containerMap: removing container" podUID="3a29a9f5-5ad9-4502-8090-576b2618c335" containerName="manager"
Oct 30 18:44:55.426082 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.426025    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a29a9f5-5ad9-4502-8090-576b2618c335" containerName="manager"
Oct 30 18:44:55.426133 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.426118    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a29a9f5-5ad9-4502-8090-576b2618c335" containerName="manager"
Oct 30 18:44:55.428890 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.428872    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq"]
Oct 30 18:44:55.428999 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.428904    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq"]
Oct 30 18:44:55.429056 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.429044    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72"
Oct 30 18:44:55.429112 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.429064    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq" podUID="005122d5-6fc8-407a-aaee-80a3e6d0d4a0" containerName="manager" containerID="cri-o://3a9dbe1dd7bf743e566ba60a64fc0f9402dd1fd70ea0ac10f7e519772805b3c9" gracePeriod=2
Oct 30 18:44:55.430409 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.430388    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-khfdk"]
Oct 30 18:44:55.430772 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.430748    4544 cpu_manager.go:414] "containerMap: removing container" podUID="005122d5-6fc8-407a-aaee-80a3e6d0d4a0" containerName="manager"
Oct 30 18:44:55.430772 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.430764    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="005122d5-6fc8-407a-aaee-80a3e6d0d4a0" containerName="manager"
Oct 30 18:44:55.430872 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.430865    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="005122d5-6fc8-407a-aaee-80a3e6d0d4a0" containerName="manager"
Oct 30 18:44:55.433591 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.433571    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-khfdk"
Oct 30 18:44:55.472492 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:44:55.472452    4544 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-847vz\" is forbidden: User \"system:node:ip-10-0-140-122.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-122.us-west-2.compute.internal' and this object" podUID="fe49dc86-2dd1-410d-a21d-1c3033ab3ae9" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"
Oct 30 18:44:55.478673 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.477297    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6cd7\" (UniqueName: \"kubernetes.io/projected/d36b1da5-c6f2-4106-88c9-345397cae78a-kube-api-access-m6cd7\") pod \"kuadrant-operator-controller-manager-6b6fb56846-zvjkx\" (UID: \"d36b1da5-c6f2-4106-88c9-345397cae78a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx"
Oct 30 18:44:55.478673 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.477362    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg4vl\" (UniqueName: \"kubernetes.io/projected/3684a66d-aa78-4d80-8229-4b32be8f7ac2-kube-api-access-xg4vl\") pod \"limitador-operator-controller-manager-856dd584c5-khfdk\" (UID: \"3684a66d-aa78-4d80-8229-4b32be8f7ac2\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-khfdk"
Oct 30 18:44:55.478673 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.477428    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d36b1da5-c6f2-4106-88c9-345397cae78a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-zvjkx\" (UID: \"d36b1da5-c6f2-4106-88c9-345397cae78a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx"
Oct 30 18:44:55.478673 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.477463    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f87df56-9199-49d1-b946-25e51fc568ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b76784d48-q7d72\" (UID: \"4f87df56-9199-49d1-b946-25e51fc568ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72"
Oct 30 18:44:55.478673 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.477492    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wb9d\" (UniqueName: \"kubernetes.io/projected/4f87df56-9199-49d1-b946-25e51fc568ef-kube-api-access-4wb9d\") pod \"kuadrant-operator-controller-manager-6b76784d48-q7d72\" (UID: \"4f87df56-9199-49d1-b946-25e51fc568ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72"
Oct 30 18:44:55.485291 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.485263    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx"]
Oct 30 18:44:55.537348 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.537310    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72"]
Oct 30 18:44:55.541771 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.541742    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-khfdk"]
Oct 30 18:44:55.577993 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.577966    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6cd7\" (UniqueName: \"kubernetes.io/projected/d36b1da5-c6f2-4106-88c9-345397cae78a-kube-api-access-m6cd7\") pod \"kuadrant-operator-controller-manager-6b6fb56846-zvjkx\" (UID: \"d36b1da5-c6f2-4106-88c9-345397cae78a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx"
Oct 30 18:44:55.578122 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.578013    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg4vl\" (UniqueName: \"kubernetes.io/projected/3684a66d-aa78-4d80-8229-4b32be8f7ac2-kube-api-access-xg4vl\") pod \"limitador-operator-controller-manager-856dd584c5-khfdk\" (UID: \"3684a66d-aa78-4d80-8229-4b32be8f7ac2\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-khfdk"
Oct 30 18:44:55.578239 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.578221    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d36b1da5-c6f2-4106-88c9-345397cae78a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-zvjkx\" (UID: \"d36b1da5-c6f2-4106-88c9-345397cae78a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx"
Oct 30 18:44:55.578278 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.578256    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f87df56-9199-49d1-b946-25e51fc568ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b76784d48-q7d72\" (UID: \"4f87df56-9199-49d1-b946-25e51fc568ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72"
Oct 30 18:44:55.578324 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.578277    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wb9d\" (UniqueName: \"kubernetes.io/projected/4f87df56-9199-49d1-b946-25e51fc568ef-kube-api-access-4wb9d\") pod \"kuadrant-operator-controller-manager-6b76784d48-q7d72\" (UID: \"4f87df56-9199-49d1-b946-25e51fc568ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72"
Oct 30 18:44:55.578679 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.578654    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d36b1da5-c6f2-4106-88c9-345397cae78a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-zvjkx\" (UID: \"d36b1da5-c6f2-4106-88c9-345397cae78a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx"
Oct 30 18:44:55.578679 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.578674    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f87df56-9199-49d1-b946-25e51fc568ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b76784d48-q7d72\" (UID: \"4f87df56-9199-49d1-b946-25e51fc568ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72"
Oct 30 18:44:55.622945 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.622925    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"
Oct 30 18:44:55.633035 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:44:55.633002    4544 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-m9qzq\" is forbidden: User \"system:node:ip-10-0-140-122.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-122.us-west-2.compute.internal' and this object" podUID="005122d5-6fc8-407a-aaee-80a3e6d0d4a0" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq"
Oct 30 18:44:55.634315 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.634295    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg4vl\" (UniqueName: \"kubernetes.io/projected/3684a66d-aa78-4d80-8229-4b32be8f7ac2-kube-api-access-xg4vl\") pod \"limitador-operator-controller-manager-856dd584c5-khfdk\" (UID: \"3684a66d-aa78-4d80-8229-4b32be8f7ac2\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-khfdk"
Oct 30 18:44:55.637638 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.637589    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6cd7\" (UniqueName: \"kubernetes.io/projected/d36b1da5-c6f2-4106-88c9-345397cae78a-kube-api-access-m6cd7\") pod \"kuadrant-operator-controller-manager-6b6fb56846-zvjkx\" (UID: \"d36b1da5-c6f2-4106-88c9-345397cae78a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx"
Oct 30 18:44:55.637978 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.637956    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wb9d\" (UniqueName: \"kubernetes.io/projected/4f87df56-9199-49d1-b946-25e51fc568ef-kube-api-access-4wb9d\") pod \"kuadrant-operator-controller-manager-6b76784d48-q7d72\" (UID: \"4f87df56-9199-49d1-b946-25e51fc568ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72"
Oct 30 18:44:55.678819 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.678790    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe49dc86-2dd1-410d-a21d-1c3033ab3ae9-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe49dc86-2dd1-410d-a21d-1c3033ab3ae9-extensions-socket-volume\") pod \"fe49dc86-2dd1-410d-a21d-1c3033ab3ae9\" (UID: \"fe49dc86-2dd1-410d-a21d-1c3033ab3ae9\") "
Oct 30 18:44:55.678977 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.678883    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe49dc86-2dd1-410d-a21d-1c3033ab3ae9-kube-api-access-ns9dn\" (UniqueName: \"kubernetes.io/projected/fe49dc86-2dd1-410d-a21d-1c3033ab3ae9-kube-api-access-ns9dn\") pod \"fe49dc86-2dd1-410d-a21d-1c3033ab3ae9\" (UID: \"fe49dc86-2dd1-410d-a21d-1c3033ab3ae9\") "
Oct 30 18:44:55.679147 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.679125    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe49dc86-2dd1-410d-a21d-1c3033ab3ae9-extensions-socket-volume" pod "fe49dc86-2dd1-410d-a21d-1c3033ab3ae9" (UID: "fe49dc86-2dd1-410d-a21d-1c3033ab3ae9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:44:55.680888 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.680864    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe49dc86-2dd1-410d-a21d-1c3033ab3ae9-kube-api-access-ns9dn" pod "fe49dc86-2dd1-410d-a21d-1c3033ab3ae9" (UID: "fe49dc86-2dd1-410d-a21d-1c3033ab3ae9"). InnerVolumeSpecName "kube-api-access-ns9dn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:44:55.711052 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.711035    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq"
Oct 30 18:44:55.727084 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.727065    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx"
Oct 30 18:44:55.781131 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.781100    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/005122d5-6fc8-407a-aaee-80a3e6d0d4a0-kube-api-access-hq5gb\" (UniqueName: \"kubernetes.io/projected/005122d5-6fc8-407a-aaee-80a3e6d0d4a0-kube-api-access-hq5gb\") pod \"005122d5-6fc8-407a-aaee-80a3e6d0d4a0\" (UID: \"005122d5-6fc8-407a-aaee-80a3e6d0d4a0\") "
Oct 30 18:44:55.781415 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.781396    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ns9dn\" (UniqueName: \"kubernetes.io/projected/fe49dc86-2dd1-410d-a21d-1c3033ab3ae9-kube-api-access-ns9dn\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:44:55.781486 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.781422    4544 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe49dc86-2dd1-410d-a21d-1c3033ab3ae9-extensions-socket-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:44:55.784200 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.784175    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/005122d5-6fc8-407a-aaee-80a3e6d0d4a0-kube-api-access-hq5gb" pod "005122d5-6fc8-407a-aaee-80a3e6d0d4a0" (UID: "005122d5-6fc8-407a-aaee-80a3e6d0d4a0"). InnerVolumeSpecName "kube-api-access-hq5gb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:44:55.787245 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:44:55.787211    4544 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-847vz\" is forbidden: User \"system:node:ip-10-0-140-122.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-122.us-west-2.compute.internal' and this object" podUID="fe49dc86-2dd1-410d-a21d-1c3033ab3ae9" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"
Oct 30 18:44:55.796282 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.796261    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72"
Oct 30 18:44:55.801411 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.801394    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-khfdk"
Oct 30 18:44:55.882172 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:55.882146    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hq5gb\" (UniqueName: \"kubernetes.io/projected/005122d5-6fc8-407a-aaee-80a3e6d0d4a0-kube-api-access-hq5gb\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:44:55.952299 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:44:55.952258    4544 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-847vz\" is forbidden: User \"system:node:ip-10-0-140-122.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-122.us-west-2.compute.internal' and this object" podUID="fe49dc86-2dd1-410d-a21d-1c3033ab3ae9" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"
Oct 30 18:44:56.000249 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:44:56.000218    4544 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-m9qzq\" is forbidden: User \"system:node:ip-10-0-140-122.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-122.us-west-2.compute.internal' and this object" podUID="005122d5-6fc8-407a-aaee-80a3e6d0d4a0" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq"
Oct 30 18:44:56.141147 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.141126    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx"]
Oct 30 18:44:56.143899 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:44:56.143872    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd36b1da5_c6f2_4106_88c9_345397cae78a.slice/crio-77dac619da8bd11bede34da28e4a46e8de3226d7044ce7a78cfa1dac5131486c WatchSource:0}: Error finding container 77dac619da8bd11bede34da28e4a46e8de3226d7044ce7a78cfa1dac5131486c: Status 404 returned error can't find the container with id 77dac619da8bd11bede34da28e4a46e8de3226d7044ce7a78cfa1dac5131486c
Oct 30 18:44:56.193531 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.193509    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-khfdk"]
Oct 30 18:44:56.197024 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.196846    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72"]
Oct 30 18:44:56.197110 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:44:56.197033    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3684a66d_aa78_4d80_8229_4b32be8f7ac2.slice/crio-d66a1040db710389744c67f51876065b57297689b14945045ca1f0e9a0270840 WatchSource:0}: Error finding container d66a1040db710389744c67f51876065b57297689b14945045ca1f0e9a0270840: Status 404 returned error can't find the container with id d66a1040db710389744c67f51876065b57297689b14945045ca1f0e9a0270840
Oct 30 18:44:56.199916 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:44:56.199894    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f87df56_9199_49d1_b946_25e51fc568ef.slice/crio-2020a20fb82c4abf5843a686f3a8305cf7c7f3730eb00952fb42460fe431d1ec WatchSource:0}: Error finding container 2020a20fb82c4abf5843a686f3a8305cf7c7f3730eb00952fb42460fe431d1ec: Status 404 returned error can't find the container with id 2020a20fb82c4abf5843a686f3a8305cf7c7f3730eb00952fb42460fe431d1ec
Oct 30 18:44:56.367799 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.367767    4544 generic.go:356] "Generic (PLEG): container finished" podID="fe49dc86-2dd1-410d-a21d-1c3033ab3ae9" containerID="40d1b5cd9a9e960a108f43fd272b53907775921da1d8e01e06253f8b223590a8" exitCode=0
Oct 30 18:44:56.368666 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.367820    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"
Oct 30 18:44:56.368666 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.367934    4544 scope.go:122] "RemoveContainer" containerID="40d1b5cd9a9e960a108f43fd272b53907775921da1d8e01e06253f8b223590a8"
Oct 30 18:44:56.370077 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.370050    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-khfdk" event={"ID":"3684a66d-aa78-4d80-8229-4b32be8f7ac2","Type":"ContainerStarted","Data":"a33249776f1e8a589fea2d5200fa969b6224d30dd74427b36267d869aae9f9de"}
Oct 30 18:44:56.370251 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.370104    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-khfdk"
Oct 30 18:44:56.370251 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.370122    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-khfdk" event={"ID":"3684a66d-aa78-4d80-8229-4b32be8f7ac2","Type":"ContainerStarted","Data":"d66a1040db710389744c67f51876065b57297689b14945045ca1f0e9a0270840"}
Oct 30 18:44:56.372024 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.371999    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx" event={"ID":"d36b1da5-c6f2-4106-88c9-345397cae78a","Type":"ContainerStarted","Data":"e6c846d60b2a8eee368d6a965b17adb2028d496199e03d1a34511c374be49f34"}
Oct 30 18:44:56.372132 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.372052    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx" event={"ID":"d36b1da5-c6f2-4106-88c9-345397cae78a","Type":"ContainerStarted","Data":"77dac619da8bd11bede34da28e4a46e8de3226d7044ce7a78cfa1dac5131486c"}
Oct 30 18:44:56.372499 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.372478    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx"
Oct 30 18:44:56.374146 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.374063    4544 generic.go:356] "Generic (PLEG): container finished" podID="005122d5-6fc8-407a-aaee-80a3e6d0d4a0" containerID="3a9dbe1dd7bf743e566ba60a64fc0f9402dd1fd70ea0ac10f7e519772805b3c9" exitCode=0
Oct 30 18:44:56.374146 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.374095    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq"
Oct 30 18:44:56.376117 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.376089    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72" event={"ID":"4f87df56-9199-49d1-b946-25e51fc568ef","Type":"ContainerStarted","Data":"39f995318e9bc9bbd2e013a068489ee58bc1822b70f71eb33aaec029a8e3a89c"}
Oct 30 18:44:56.376211 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.376114    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm" podUID="3a29a9f5-5ad9-4502-8090-576b2618c335" containerName="manager" containerID="cri-o://7b76c0ceb4a9652a7cdc40a5075c5b532528d6a16e277beb0350be107159dd8e" gracePeriod=2
Oct 30 18:44:56.376211 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.376138    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72" event={"ID":"4f87df56-9199-49d1-b946-25e51fc568ef","Type":"ContainerStarted","Data":"2020a20fb82c4abf5843a686f3a8305cf7c7f3730eb00952fb42460fe431d1ec"}
Oct 30 18:44:56.384167 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.384149    4544 scope.go:122] "RemoveContainer" containerID="40d1b5cd9a9e960a108f43fd272b53907775921da1d8e01e06253f8b223590a8"
Oct 30 18:44:56.384502 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:44:56.384428    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40d1b5cd9a9e960a108f43fd272b53907775921da1d8e01e06253f8b223590a8\": container with ID starting with 40d1b5cd9a9e960a108f43fd272b53907775921da1d8e01e06253f8b223590a8 not found: ID does not exist" containerID="40d1b5cd9a9e960a108f43fd272b53907775921da1d8e01e06253f8b223590a8"
Oct 30 18:44:56.384502 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.384453    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40d1b5cd9a9e960a108f43fd272b53907775921da1d8e01e06253f8b223590a8"} err="failed to get container status \"40d1b5cd9a9e960a108f43fd272b53907775921da1d8e01e06253f8b223590a8\": rpc error: code = NotFound desc = could not find container \"40d1b5cd9a9e960a108f43fd272b53907775921da1d8e01e06253f8b223590a8\": container with ID starting with 40d1b5cd9a9e960a108f43fd272b53907775921da1d8e01e06253f8b223590a8 not found: ID does not exist"
Oct 30 18:44:56.384502 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.384470    4544 scope.go:122] "RemoveContainer" containerID="3a9dbe1dd7bf743e566ba60a64fc0f9402dd1fd70ea0ac10f7e519772805b3c9"
Oct 30 18:44:56.392374 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.392353    4544 scope.go:122] "RemoveContainer" containerID="3a9dbe1dd7bf743e566ba60a64fc0f9402dd1fd70ea0ac10f7e519772805b3c9"
Oct 30 18:44:56.392634 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:44:56.392612    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a9dbe1dd7bf743e566ba60a64fc0f9402dd1fd70ea0ac10f7e519772805b3c9\": container with ID starting with 3a9dbe1dd7bf743e566ba60a64fc0f9402dd1fd70ea0ac10f7e519772805b3c9 not found: ID does not exist" containerID="3a9dbe1dd7bf743e566ba60a64fc0f9402dd1fd70ea0ac10f7e519772805b3c9"
Oct 30 18:44:56.392731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.392642    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a9dbe1dd7bf743e566ba60a64fc0f9402dd1fd70ea0ac10f7e519772805b3c9"} err="failed to get container status \"3a9dbe1dd7bf743e566ba60a64fc0f9402dd1fd70ea0ac10f7e519772805b3c9\": rpc error: code = NotFound desc = could not find container \"3a9dbe1dd7bf743e566ba60a64fc0f9402dd1fd70ea0ac10f7e519772805b3c9\": container with ID starting with 3a9dbe1dd7bf743e566ba60a64fc0f9402dd1fd70ea0ac10f7e519772805b3c9 not found: ID does not exist"
Oct 30 18:44:56.415216 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:44:56.415190    4544 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-m9qzq\" is forbidden: User \"system:node:ip-10-0-140-122.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-122.us-west-2.compute.internal' and this object" podUID="005122d5-6fc8-407a-aaee-80a3e6d0d4a0" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq"
Oct 30 18:44:56.462255 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:44:56.462231    4544 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-847vz\" is forbidden: User \"system:node:ip-10-0-140-122.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-122.us-west-2.compute.internal' and this object" podUID="fe49dc86-2dd1-410d-a21d-1c3033ab3ae9" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"
Oct 30 18:44:56.577545 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.577498    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72" podStartSLOduration=1.577483554 podStartE2EDuration="1.577483554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:56.576245897 +0000 UTC m=+494.102120992" watchObservedRunningTime="2026-10-30 18:44:56.577483554 +0000 UTC m=+494.103358649"
Oct 30 18:44:56.604931 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.604895    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="005122d5-6fc8-407a-aaee-80a3e6d0d4a0" path="/var/lib/kubelet/pods/005122d5-6fc8-407a-aaee-80a3e6d0d4a0/volumes"
Oct 30 18:44:56.605448 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.605426    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe49dc86-2dd1-410d-a21d-1c3033ab3ae9" path="/var/lib/kubelet/pods/fe49dc86-2dd1-410d-a21d-1c3033ab3ae9/volumes"
Oct 30 18:44:56.623266 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:44:56.623235    4544 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-m9qzq\" is forbidden: User \"system:node:ip-10-0-140-122.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-122.us-west-2.compute.internal' and this object" podUID="005122d5-6fc8-407a-aaee-80a3e6d0d4a0" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-m9qzq"
Oct 30 18:44:56.663874 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.663845    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm"
Oct 30 18:44:56.689017 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.688990    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a29a9f5-5ad9-4502-8090-576b2618c335-kube-api-access-ztkpv\" (UniqueName: \"kubernetes.io/projected/3a29a9f5-5ad9-4502-8090-576b2618c335-kube-api-access-ztkpv\") pod \"3a29a9f5-5ad9-4502-8090-576b2618c335\" (UID: \"3a29a9f5-5ad9-4502-8090-576b2618c335\") "
Oct 30 18:44:56.689144 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.689042    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a29a9f5-5ad9-4502-8090-576b2618c335-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a29a9f5-5ad9-4502-8090-576b2618c335-extensions-socket-volume\") pod \"3a29a9f5-5ad9-4502-8090-576b2618c335\" (UID: \"3a29a9f5-5ad9-4502-8090-576b2618c335\") "
Oct 30 18:44:56.689400 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.689377    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a29a9f5-5ad9-4502-8090-576b2618c335-extensions-socket-volume" pod "3a29a9f5-5ad9-4502-8090-576b2618c335" (UID: "3a29a9f5-5ad9-4502-8090-576b2618c335"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:44:56.690950 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.690929    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a29a9f5-5ad9-4502-8090-576b2618c335-kube-api-access-ztkpv" pod "3a29a9f5-5ad9-4502-8090-576b2618c335" (UID: "3a29a9f5-5ad9-4502-8090-576b2618c335"). InnerVolumeSpecName "kube-api-access-ztkpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:44:56.735118 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.735078    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-khfdk" podStartSLOduration=1.735065991 podStartE2EDuration="1.735065991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:56.734102476 +0000 UTC m=+494.259977573" watchObservedRunningTime="2026-10-30 18:44:56.735065991 +0000 UTC m=+494.260941086"
Oct 30 18:44:56.781188 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:44:56.781152    4544 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-847vz\" is forbidden: User \"system:node:ip-10-0-140-122.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-122.us-west-2.compute.internal' and this object" podUID="fe49dc86-2dd1-410d-a21d-1c3033ab3ae9" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-847vz"
Oct 30 18:44:56.789403 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.789381    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztkpv\" (UniqueName: \"kubernetes.io/projected/3a29a9f5-5ad9-4502-8090-576b2618c335-kube-api-access-ztkpv\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:44:56.789478 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.789407    4544 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a29a9f5-5ad9-4502-8090-576b2618c335-extensions-socket-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:44:56.828785 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:44:56.828756    4544 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b76784d48-8rhnm\" is forbidden: User \"system:node:ip-10-0-140-122.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-122.us-west-2.compute.internal' and this object" podUID="3a29a9f5-5ad9-4502-8090-576b2618c335" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm"
Oct 30 18:44:56.943447 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:56.943352    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx" podStartSLOduration=1.943335346 podStartE2EDuration="1.943335346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:56.941965157 +0000 UTC m=+494.467840253" watchObservedRunningTime="2026-10-30 18:44:56.943335346 +0000 UTC m=+494.469210441"
Oct 30 18:44:57.384208 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:57.384167    4544 generic.go:356] "Generic (PLEG): container finished" podID="3a29a9f5-5ad9-4502-8090-576b2618c335" containerID="7b76c0ceb4a9652a7cdc40a5075c5b532528d6a16e277beb0350be107159dd8e" exitCode=0
Oct 30 18:44:57.384613 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:57.384213    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm"
Oct 30 18:44:57.384613 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:57.384260    4544 scope.go:122] "RemoveContainer" containerID="7b76c0ceb4a9652a7cdc40a5075c5b532528d6a16e277beb0350be107159dd8e"
Oct 30 18:44:57.384862 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:57.384842    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72"
Oct 30 18:44:57.396786 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:57.396768    4544 scope.go:122] "RemoveContainer" containerID="7b76c0ceb4a9652a7cdc40a5075c5b532528d6a16e277beb0350be107159dd8e"
Oct 30 18:44:57.397065 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:44:57.397041    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b76c0ceb4a9652a7cdc40a5075c5b532528d6a16e277beb0350be107159dd8e\": container with ID starting with 7b76c0ceb4a9652a7cdc40a5075c5b532528d6a16e277beb0350be107159dd8e not found: ID does not exist" containerID="7b76c0ceb4a9652a7cdc40a5075c5b532528d6a16e277beb0350be107159dd8e"
Oct 30 18:44:57.397127 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:57.397072    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b76c0ceb4a9652a7cdc40a5075c5b532528d6a16e277beb0350be107159dd8e"} err="failed to get container status \"7b76c0ceb4a9652a7cdc40a5075c5b532528d6a16e277beb0350be107159dd8e\": rpc error: code = NotFound desc = could not find container \"7b76c0ceb4a9652a7cdc40a5075c5b532528d6a16e277beb0350be107159dd8e\": container with ID starting with 7b76c0ceb4a9652a7cdc40a5075c5b532528d6a16e277beb0350be107159dd8e not found: ID does not exist"
Oct 30 18:44:57.442936 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:44:57.442880    4544 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b76784d48-8rhnm\" is forbidden: User \"system:node:ip-10-0-140-122.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-122.us-west-2.compute.internal' and this object" podUID="3a29a9f5-5ad9-4502-8090-576b2618c335" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-8rhnm"
Oct 30 18:44:58.600975 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:44:58.600941    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a29a9f5-5ad9-4502-8090-576b2618c335" path="/var/lib/kubelet/pods/3a29a9f5-5ad9-4502-8090-576b2618c335/volumes"
Oct 30 18:45:05.530296 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:45:05.530258    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:45:07.387480 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:07.387442    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-khfdk"
Oct 30 18:45:08.391697 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:08.391663    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx"
Oct 30 18:45:08.392077 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:08.391736    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72"
Oct 30 18:45:08.694060 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:08.693985    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx"]
Oct 30 18:45:08.694202 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:08.694168    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx" podUID="d36b1da5-c6f2-4106-88c9-345397cae78a" containerName="manager" containerID="cri-o://e6c846d60b2a8eee368d6a965b17adb2028d496199e03d1a34511c374be49f34" gracePeriod=10
Oct 30 18:45:08.947100 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:08.947045    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65"]
Oct 30 18:45:08.961529 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:08.961508    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx"
Oct 30 18:45:08.971679 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:08.971659    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65"
Oct 30 18:45:09.010818 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.010791    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65"]
Oct 30 18:45:09.085385 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.085357    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d36b1da5-c6f2-4106-88c9-345397cae78a-kube-api-access-m6cd7\" (UniqueName: \"kubernetes.io/projected/d36b1da5-c6f2-4106-88c9-345397cae78a-kube-api-access-m6cd7\") pod \"d36b1da5-c6f2-4106-88c9-345397cae78a\" (UID: \"d36b1da5-c6f2-4106-88c9-345397cae78a\") "
Oct 30 18:45:09.085505 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.085451    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d36b1da5-c6f2-4106-88c9-345397cae78a-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d36b1da5-c6f2-4106-88c9-345397cae78a-extensions-socket-volume\") pod \"d36b1da5-c6f2-4106-88c9-345397cae78a\" (UID: \"d36b1da5-c6f2-4106-88c9-345397cae78a\") "
Oct 30 18:45:09.085596 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.085578    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28jlz\" (UniqueName: \"kubernetes.io/projected/d78e6841-0805-4163-a3cd-830833d24a2a-kube-api-access-28jlz\") pod \"kuadrant-operator-controller-manager-57bb9bfff-w6t65\" (UID: \"d78e6841-0805-4163-a3cd-830833d24a2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65"
Oct 30 18:45:09.085775 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.085754    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d78e6841-0805-4163-a3cd-830833d24a2a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-w6t65\" (UID: \"d78e6841-0805-4163-a3cd-830833d24a2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65"
Oct 30 18:45:09.085845 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.085801    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d36b1da5-c6f2-4106-88c9-345397cae78a-extensions-socket-volume" pod "d36b1da5-c6f2-4106-88c9-345397cae78a" (UID: "d36b1da5-c6f2-4106-88c9-345397cae78a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:45:09.087339 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.087316    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d36b1da5-c6f2-4106-88c9-345397cae78a-kube-api-access-m6cd7" pod "d36b1da5-c6f2-4106-88c9-345397cae78a" (UID: "d36b1da5-c6f2-4106-88c9-345397cae78a"). InnerVolumeSpecName "kube-api-access-m6cd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:09.187009 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.186981    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d78e6841-0805-4163-a3cd-830833d24a2a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-w6t65\" (UID: \"d78e6841-0805-4163-a3cd-830833d24a2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65"
Oct 30 18:45:09.187143 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.187013    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28jlz\" (UniqueName: \"kubernetes.io/projected/d78e6841-0805-4163-a3cd-830833d24a2a-kube-api-access-28jlz\") pod \"kuadrant-operator-controller-manager-57bb9bfff-w6t65\" (UID: \"d78e6841-0805-4163-a3cd-830833d24a2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65"
Oct 30 18:45:09.187143 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.187086    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6cd7\" (UniqueName: \"kubernetes.io/projected/d36b1da5-c6f2-4106-88c9-345397cae78a-kube-api-access-m6cd7\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:45:09.187143 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.187097    4544 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d36b1da5-c6f2-4106-88c9-345397cae78a-extensions-socket-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:45:09.187432 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.187410    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d78e6841-0805-4163-a3cd-830833d24a2a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-w6t65\" (UID: \"d78e6841-0805-4163-a3cd-830833d24a2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65"
Oct 30 18:45:09.241212 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.241158    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28jlz\" (UniqueName: \"kubernetes.io/projected/d78e6841-0805-4163-a3cd-830833d24a2a-kube-api-access-28jlz\") pod \"kuadrant-operator-controller-manager-57bb9bfff-w6t65\" (UID: \"d78e6841-0805-4163-a3cd-830833d24a2a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65"
Oct 30 18:45:09.293985 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.293956    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65"
Oct 30 18:45:09.433478 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.433450    4544 generic.go:356] "Generic (PLEG): container finished" podID="d36b1da5-c6f2-4106-88c9-345397cae78a" containerID="e6c846d60b2a8eee368d6a965b17adb2028d496199e03d1a34511c374be49f34" exitCode=0
Oct 30 18:45:09.433892 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.433487    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx" event={"ID":"d36b1da5-c6f2-4106-88c9-345397cae78a","Type":"ContainerDied","Data":"e6c846d60b2a8eee368d6a965b17adb2028d496199e03d1a34511c374be49f34"}
Oct 30 18:45:09.433892 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.433514    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx"
Oct 30 18:45:09.433892 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.433548    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx" event={"ID":"d36b1da5-c6f2-4106-88c9-345397cae78a","Type":"ContainerDied","Data":"77dac619da8bd11bede34da28e4a46e8de3226d7044ce7a78cfa1dac5131486c"}
Oct 30 18:45:09.433892 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.433570    4544 scope.go:122] "RemoveContainer" containerID="e6c846d60b2a8eee368d6a965b17adb2028d496199e03d1a34511c374be49f34"
Oct 30 18:45:09.441515 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.441497    4544 scope.go:122] "RemoveContainer" containerID="e6c846d60b2a8eee368d6a965b17adb2028d496199e03d1a34511c374be49f34"
Oct 30 18:45:09.441807 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:45:09.441774    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6c846d60b2a8eee368d6a965b17adb2028d496199e03d1a34511c374be49f34\": container with ID starting with e6c846d60b2a8eee368d6a965b17adb2028d496199e03d1a34511c374be49f34 not found: ID does not exist" containerID="e6c846d60b2a8eee368d6a965b17adb2028d496199e03d1a34511c374be49f34"
Oct 30 18:45:09.441881 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.441804    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6c846d60b2a8eee368d6a965b17adb2028d496199e03d1a34511c374be49f34"} err="failed to get container status \"e6c846d60b2a8eee368d6a965b17adb2028d496199e03d1a34511c374be49f34\": rpc error: code = NotFound desc = could not find container \"e6c846d60b2a8eee368d6a965b17adb2028d496199e03d1a34511c374be49f34\": container with ID starting with e6c846d60b2a8eee368d6a965b17adb2028d496199e03d1a34511c374be49f34 not found: ID does not exist"
Oct 30 18:45:09.587211 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.587189    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx"]
Oct 30 18:45:09.591903 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.591885    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-zvjkx"]
Oct 30 18:45:09.747377 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:09.747344    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65"]
Oct 30 18:45:09.751318 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:45:09.751294    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd78e6841_0805_4163_a3cd_830833d24a2a.slice/crio-62134ca4732528cfb2e4bae4ccf55502e31fa54821cf1eeb92a61c7a3f8afe1e WatchSource:0}: Error finding container 62134ca4732528cfb2e4bae4ccf55502e31fa54821cf1eeb92a61c7a3f8afe1e: Status 404 returned error can't find the container with id 62134ca4732528cfb2e4bae4ccf55502e31fa54821cf1eeb92a61c7a3f8afe1e
Oct 30 18:45:10.439733 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:10.439697    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65" event={"ID":"d78e6841-0805-4163-a3cd-830833d24a2a","Type":"ContainerStarted","Data":"69f0cb4e4901e092eb71d4f327245754a8145f2fbb789d2424c6d0b2bb7f5b49"}
Oct 30 18:45:10.440107 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:10.439749    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65" event={"ID":"d78e6841-0805-4163-a3cd-830833d24a2a","Type":"ContainerStarted","Data":"62134ca4732528cfb2e4bae4ccf55502e31fa54821cf1eeb92a61c7a3f8afe1e"}
Oct 30 18:45:10.440107 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:10.439837    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65"
Oct 30 18:45:10.549945 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:10.549896    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65" podStartSLOduration=2.549884511 podStartE2EDuration="2.549884511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:10.549285482 +0000 UTC m=+508.075160576" watchObservedRunningTime="2026-10-30 18:45:10.549884511 +0000 UTC m=+508.075759608"
Oct 30 18:45:10.600951 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:10.600923    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d36b1da5-c6f2-4106-88c9-345397cae78a" path="/var/lib/kubelet/pods/d36b1da5-c6f2-4106-88c9-345397cae78a/volumes"
Oct 30 18:45:21.445332 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:21.445301    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65"
Oct 30 18:45:21.648463 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:21.648430    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72"]
Oct 30 18:45:21.648684 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:21.648647    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72" podUID="4f87df56-9199-49d1-b946-25e51fc568ef" containerName="manager" containerID="cri-o://39f995318e9bc9bbd2e013a068489ee58bc1822b70f71eb33aaec029a8e3a89c" gracePeriod=10
Oct 30 18:45:21.889962 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:21.889938    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72"
Oct 30 18:45:21.989583 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:21.989552    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f87df56-9199-49d1-b946-25e51fc568ef-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f87df56-9199-49d1-b946-25e51fc568ef-extensions-socket-volume\") pod \"4f87df56-9199-49d1-b946-25e51fc568ef\" (UID: \"4f87df56-9199-49d1-b946-25e51fc568ef\") "
Oct 30 18:45:21.989731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:21.989662    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f87df56-9199-49d1-b946-25e51fc568ef-kube-api-access-4wb9d\" (UniqueName: \"kubernetes.io/projected/4f87df56-9199-49d1-b946-25e51fc568ef-kube-api-access-4wb9d\") pod \"4f87df56-9199-49d1-b946-25e51fc568ef\" (UID: \"4f87df56-9199-49d1-b946-25e51fc568ef\") "
Oct 30 18:45:21.989969 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:21.989944    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f87df56-9199-49d1-b946-25e51fc568ef-extensions-socket-volume" pod "4f87df56-9199-49d1-b946-25e51fc568ef" (UID: "4f87df56-9199-49d1-b946-25e51fc568ef"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:45:21.991557 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:21.991539    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f87df56-9199-49d1-b946-25e51fc568ef-kube-api-access-4wb9d" pod "4f87df56-9199-49d1-b946-25e51fc568ef" (UID: "4f87df56-9199-49d1-b946-25e51fc568ef"). InnerVolumeSpecName "kube-api-access-4wb9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:22.090192 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:22.090162    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wb9d\" (UniqueName: \"kubernetes.io/projected/4f87df56-9199-49d1-b946-25e51fc568ef-kube-api-access-4wb9d\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:45:22.090192 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:22.090186    4544 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f87df56-9199-49d1-b946-25e51fc568ef-extensions-socket-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:45:22.492783 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:22.492690    4544 generic.go:356] "Generic (PLEG): container finished" podID="4f87df56-9199-49d1-b946-25e51fc568ef" containerID="39f995318e9bc9bbd2e013a068489ee58bc1822b70f71eb33aaec029a8e3a89c" exitCode=0
Oct 30 18:45:22.492783 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:22.492745    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72" event={"ID":"4f87df56-9199-49d1-b946-25e51fc568ef","Type":"ContainerDied","Data":"39f995318e9bc9bbd2e013a068489ee58bc1822b70f71eb33aaec029a8e3a89c"}
Oct 30 18:45:22.492783 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:22.492753    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72"
Oct 30 18:45:22.492783 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:22.492782    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72" event={"ID":"4f87df56-9199-49d1-b946-25e51fc568ef","Type":"ContainerDied","Data":"2020a20fb82c4abf5843a686f3a8305cf7c7f3730eb00952fb42460fe431d1ec"}
Oct 30 18:45:22.493382 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:22.492857    4544 scope.go:122] "RemoveContainer" containerID="39f995318e9bc9bbd2e013a068489ee58bc1822b70f71eb33aaec029a8e3a89c"
Oct 30 18:45:22.500792 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:22.500770    4544 scope.go:122] "RemoveContainer" containerID="39f995318e9bc9bbd2e013a068489ee58bc1822b70f71eb33aaec029a8e3a89c"
Oct 30 18:45:22.501043 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:45:22.501025    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39f995318e9bc9bbd2e013a068489ee58bc1822b70f71eb33aaec029a8e3a89c\": container with ID starting with 39f995318e9bc9bbd2e013a068489ee58bc1822b70f71eb33aaec029a8e3a89c not found: ID does not exist" containerID="39f995318e9bc9bbd2e013a068489ee58bc1822b70f71eb33aaec029a8e3a89c"
Oct 30 18:45:22.501091 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:22.501053    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39f995318e9bc9bbd2e013a068489ee58bc1822b70f71eb33aaec029a8e3a89c"} err="failed to get container status \"39f995318e9bc9bbd2e013a068489ee58bc1822b70f71eb33aaec029a8e3a89c\": rpc error: code = NotFound desc = could not find container \"39f995318e9bc9bbd2e013a068489ee58bc1822b70f71eb33aaec029a8e3a89c\": container with ID starting with 39f995318e9bc9bbd2e013a068489ee58bc1822b70f71eb33aaec029a8e3a89c not found: ID does not exist"
Oct 30 18:45:22.695857 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:22.695829    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72"]
Oct 30 18:45:22.742367 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:22.742339    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q7d72"]
Oct 30 18:45:24.600164 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:24.600133    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4f87df56-9199-49d1-b946-25e51fc568ef" path="/var/lib/kubelet/pods/4f87df56-9199-49d1-b946-25e51fc568ef/volumes"
Oct 30 18:45:35.530206 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:45:35.530164    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:45:47.121290 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.121254    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-zjcch"]
Oct 30 18:45:47.121725 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.121527    4544 cpu_manager.go:414] "containerMap: removing container" podUID="4f87df56-9199-49d1-b946-25e51fc568ef" containerName="manager"
Oct 30 18:45:47.121725 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.121538    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4f87df56-9199-49d1-b946-25e51fc568ef" containerName="manager"
Oct 30 18:45:47.121725 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.121550    4544 cpu_manager.go:414] "containerMap: removing container" podUID="d36b1da5-c6f2-4106-88c9-345397cae78a" containerName="manager"
Oct 30 18:45:47.121725 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.121555    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d36b1da5-c6f2-4106-88c9-345397cae78a" containerName="manager"
Oct 30 18:45:47.121725 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.121629    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="4f87df56-9199-49d1-b946-25e51fc568ef" containerName="manager"
Oct 30 18:45:47.121725 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.121638    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="d36b1da5-c6f2-4106-88c9-345397cae78a" containerName="manager"
Oct 30 18:45:47.125858 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.125836    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-zjcch"
Oct 30 18:45:47.176581 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.176561    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 18:45:47.177393 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.177367    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4vb7\" (UniqueName: \"kubernetes.io/projected/d10eef13-c2fa-4f17-aa53-b5df498116bc-kube-api-access-h4vb7\") pod \"limitador-limitador-74dfcfb894-zjcch\" (UID: \"d10eef13-c2fa-4f17-aa53-b5df498116bc\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-zjcch"
Oct 30 18:45:47.177499 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.177458    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d10eef13-c2fa-4f17-aa53-b5df498116bc-config-file\") pod \"limitador-limitador-74dfcfb894-zjcch\" (UID: \"d10eef13-c2fa-4f17-aa53-b5df498116bc\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-zjcch"
Oct 30 18:45:47.223011 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.222986    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-szvzq\""
Oct 30 18:45:47.224028 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.224007    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-zjcch"]
Oct 30 18:45:47.278322 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.278292    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d10eef13-c2fa-4f17-aa53-b5df498116bc-config-file\") pod \"limitador-limitador-74dfcfb894-zjcch\" (UID: \"d10eef13-c2fa-4f17-aa53-b5df498116bc\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-zjcch"
Oct 30 18:45:47.278322 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.278325    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4vb7\" (UniqueName: \"kubernetes.io/projected/d10eef13-c2fa-4f17-aa53-b5df498116bc-kube-api-access-h4vb7\") pod \"limitador-limitador-74dfcfb894-zjcch\" (UID: \"d10eef13-c2fa-4f17-aa53-b5df498116bc\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-zjcch"
Oct 30 18:45:47.278953 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.278931    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d10eef13-c2fa-4f17-aa53-b5df498116bc-config-file\") pod \"limitador-limitador-74dfcfb894-zjcch\" (UID: \"d10eef13-c2fa-4f17-aa53-b5df498116bc\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-zjcch"
Oct 30 18:45:47.333817 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.333785    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4vb7\" (UniqueName: \"kubernetes.io/projected/d10eef13-c2fa-4f17-aa53-b5df498116bc-kube-api-access-h4vb7\") pod \"limitador-limitador-74dfcfb894-zjcch\" (UID: \"d10eef13-c2fa-4f17-aa53-b5df498116bc\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-zjcch"
Oct 30 18:45:47.362569 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.362544    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-zjcch"]
Oct 30 18:45:47.438347 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.438272    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-zjcch"
Oct 30 18:45:47.803306 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:47.803279    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-zjcch"]
Oct 30 18:45:47.806705 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:45:47.806677    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd10eef13_c2fa_4f17_aa53_b5df498116bc.slice/crio-fe47e6698d8560421827741a637ebe5c3cbf115304030a5415c8d8d8d7fe1411 WatchSource:0}: Error finding container fe47e6698d8560421827741a637ebe5c3cbf115304030a5415c8d8d8d7fe1411: Status 404 returned error can't find the container with id fe47e6698d8560421827741a637ebe5c3cbf115304030a5415c8d8d8d7fe1411
Oct 30 18:45:48.600283 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:48.600251    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-zjcch" event={"ID":"d10eef13-c2fa-4f17-aa53-b5df498116bc","Type":"ContainerStarted","Data":"fe47e6698d8560421827741a637ebe5c3cbf115304030a5415c8d8d8d7fe1411"}
Oct 30 18:45:50.606865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:50.606836    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-zjcch" event={"ID":"d10eef13-c2fa-4f17-aa53-b5df498116bc","Type":"ContainerStarted","Data":"20bf03298848e551ef775ee47429307257c181df04237ee20e8b29d41026e5e6"}
Oct 30 18:45:50.607184 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:50.606903    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-zjcch"
Oct 30 18:45:50.715541 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:45:50.715418    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-zjcch" podStartSLOduration=1.066188186 podStartE2EDuration="3.715403219s" totalImagesPullingTime="2.649215033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:50.71440488 +0000 UTC m=+548.240279974" watchObservedRunningTime="2026-10-30 18:45:50.715403219 +0000 UTC m=+548.241278313"
Oct 30 18:46:01.611817 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:01.611781    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-zjcch"
Oct 30 18:46:04.787047 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:04.787007    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-zjcch"]
Oct 30 18:46:04.787523 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:04.787239    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-zjcch" podUID="d10eef13-c2fa-4f17-aa53-b5df498116bc" containerName="limitador" containerID="cri-o://20bf03298848e551ef775ee47429307257c181df04237ee20e8b29d41026e5e6" gracePeriod=30
Oct 30 18:46:05.327165 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:05.327143    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-zjcch"
Oct 30 18:46:05.403717 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:05.403654    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d10eef13-c2fa-4f17-aa53-b5df498116bc-config-file\" (UniqueName: \"kubernetes.io/configmap/d10eef13-c2fa-4f17-aa53-b5df498116bc-config-file\") pod \"d10eef13-c2fa-4f17-aa53-b5df498116bc\" (UID: \"d10eef13-c2fa-4f17-aa53-b5df498116bc\") "
Oct 30 18:46:05.403717 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:05.403707    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d10eef13-c2fa-4f17-aa53-b5df498116bc-kube-api-access-h4vb7\" (UniqueName: \"kubernetes.io/projected/d10eef13-c2fa-4f17-aa53-b5df498116bc-kube-api-access-h4vb7\") pod \"d10eef13-c2fa-4f17-aa53-b5df498116bc\" (UID: \"d10eef13-c2fa-4f17-aa53-b5df498116bc\") "
Oct 30 18:46:05.404007 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:05.403983    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d10eef13-c2fa-4f17-aa53-b5df498116bc-config-file" pod "d10eef13-c2fa-4f17-aa53-b5df498116bc" (UID: "d10eef13-c2fa-4f17-aa53-b5df498116bc"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:46:05.405773 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:05.405753    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d10eef13-c2fa-4f17-aa53-b5df498116bc-kube-api-access-h4vb7" pod "d10eef13-c2fa-4f17-aa53-b5df498116bc" (UID: "d10eef13-c2fa-4f17-aa53-b5df498116bc"). InnerVolumeSpecName "kube-api-access-h4vb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:05.504560 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:05.504530    4544 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d10eef13-c2fa-4f17-aa53-b5df498116bc-config-file\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:46:05.504560 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:05.504556    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4vb7\" (UniqueName: \"kubernetes.io/projected/d10eef13-c2fa-4f17-aa53-b5df498116bc-kube-api-access-h4vb7\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:46:05.530030 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:46:05.529989    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:46:05.657998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:05.657851    4544 generic.go:356] "Generic (PLEG): container finished" podID="d10eef13-c2fa-4f17-aa53-b5df498116bc" containerID="20bf03298848e551ef775ee47429307257c181df04237ee20e8b29d41026e5e6" exitCode=0
Oct 30 18:46:05.657998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:05.657887    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-zjcch" event={"ID":"d10eef13-c2fa-4f17-aa53-b5df498116bc","Type":"ContainerDied","Data":"20bf03298848e551ef775ee47429307257c181df04237ee20e8b29d41026e5e6"}
Oct 30 18:46:05.657998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:05.657956    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-zjcch" event={"ID":"d10eef13-c2fa-4f17-aa53-b5df498116bc","Type":"ContainerDied","Data":"fe47e6698d8560421827741a637ebe5c3cbf115304030a5415c8d8d8d7fe1411"}
Oct 30 18:46:05.657998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:05.657902    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-zjcch"
Oct 30 18:46:05.657998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:05.658001    4544 scope.go:122] "RemoveContainer" containerID="20bf03298848e551ef775ee47429307257c181df04237ee20e8b29d41026e5e6"
Oct 30 18:46:05.673570 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:05.666612    4544 scope.go:122] "RemoveContainer" containerID="20bf03298848e551ef775ee47429307257c181df04237ee20e8b29d41026e5e6"
Oct 30 18:46:05.673570 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:46:05.666859    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20bf03298848e551ef775ee47429307257c181df04237ee20e8b29d41026e5e6\": container with ID starting with 20bf03298848e551ef775ee47429307257c181df04237ee20e8b29d41026e5e6 not found: ID does not exist" containerID="20bf03298848e551ef775ee47429307257c181df04237ee20e8b29d41026e5e6"
Oct 30 18:46:05.673570 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:05.666884    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20bf03298848e551ef775ee47429307257c181df04237ee20e8b29d41026e5e6"} err="failed to get container status \"20bf03298848e551ef775ee47429307257c181df04237ee20e8b29d41026e5e6\": rpc error: code = NotFound desc = could not find container \"20bf03298848e551ef775ee47429307257c181df04237ee20e8b29d41026e5e6\": container with ID starting with 20bf03298848e551ef775ee47429307257c181df04237ee20e8b29d41026e5e6 not found: ID does not exist"
Oct 30 18:46:05.813282 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:05.813255    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-zjcch"]
Oct 30 18:46:05.859235 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:05.859200    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-zjcch"]
Oct 30 18:46:06.600573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:06.600541    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d10eef13-c2fa-4f17-aa53-b5df498116bc" path="/var/lib/kubelet/pods/d10eef13-c2fa-4f17-aa53-b5df498116bc/volumes"
Oct 30 18:46:12.496590 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:12.496557    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-9zm7n"]
Oct 30 18:46:12.496977 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:12.496888    4544 cpu_manager.go:414] "containerMap: removing container" podUID="d10eef13-c2fa-4f17-aa53-b5df498116bc" containerName="limitador"
Oct 30 18:46:12.496977 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:12.496899    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d10eef13-c2fa-4f17-aa53-b5df498116bc" containerName="limitador"
Oct 30 18:46:12.496977 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:12.496951    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="d10eef13-c2fa-4f17-aa53-b5df498116bc" containerName="limitador"
Oct 30 18:46:12.499934 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:12.499916    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9zm7n"
Oct 30 18:46:12.554543 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:12.554520    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 18:46:12.554543 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:12.554526    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 18:46:12.557899 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:12.557872    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d2f0a477-b0d0-4779-9b17-4ce095ab1347-data\") pod \"postgres-7cdbf6dc44-9zm7n\" (UID: \"d2f0a477-b0d0-4779-9b17-4ce095ab1347\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9zm7n"
Oct 30 18:46:12.557998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:12.557918    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsbng\" (UniqueName: \"kubernetes.io/projected/d2f0a477-b0d0-4779-9b17-4ce095ab1347-kube-api-access-nsbng\") pod \"postgres-7cdbf6dc44-9zm7n\" (UID: \"d2f0a477-b0d0-4779-9b17-4ce095ab1347\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9zm7n"
Oct 30 18:46:12.600544 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:12.600518    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-ddl4l\""
Oct 30 18:46:12.600686 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:12.600548    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 18:46:12.601694 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:12.601677    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-9zm7n"]
Oct 30 18:46:12.658409 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:12.658383    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsbng\" (UniqueName: \"kubernetes.io/projected/d2f0a477-b0d0-4779-9b17-4ce095ab1347-kube-api-access-nsbng\") pod \"postgres-7cdbf6dc44-9zm7n\" (UID: \"d2f0a477-b0d0-4779-9b17-4ce095ab1347\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9zm7n"
Oct 30 18:46:12.659015 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:12.658956    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d2f0a477-b0d0-4779-9b17-4ce095ab1347-data\") pod \"postgres-7cdbf6dc44-9zm7n\" (UID: \"d2f0a477-b0d0-4779-9b17-4ce095ab1347\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9zm7n"
Oct 30 18:46:12.659473 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:12.659446    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d2f0a477-b0d0-4779-9b17-4ce095ab1347-data\") pod \"postgres-7cdbf6dc44-9zm7n\" (UID: \"d2f0a477-b0d0-4779-9b17-4ce095ab1347\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9zm7n"
Oct 30 18:46:12.716382 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:12.716357    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsbng\" (UniqueName: \"kubernetes.io/projected/d2f0a477-b0d0-4779-9b17-4ce095ab1347-kube-api-access-nsbng\") pod \"postgres-7cdbf6dc44-9zm7n\" (UID: \"d2f0a477-b0d0-4779-9b17-4ce095ab1347\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9zm7n"
Oct 30 18:46:12.811788 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:12.811769    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9zm7n"
Oct 30 18:46:13.171558 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:13.171472    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-9zm7n"]
Oct 30 18:46:13.174504 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:46:13.174476    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2f0a477_b0d0_4779_9b17_4ce095ab1347.slice/crio-f3787b37fc4a2361c87995358f20ec505db3ba63a1e6bcbcf263bc3e9ab11483 WatchSource:0}: Error finding container f3787b37fc4a2361c87995358f20ec505db3ba63a1e6bcbcf263bc3e9ab11483: Status 404 returned error can't find the container with id f3787b37fc4a2361c87995358f20ec505db3ba63a1e6bcbcf263bc3e9ab11483
Oct 30 18:46:13.690293 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:13.690260    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9zm7n" event={"ID":"d2f0a477-b0d0-4779-9b17-4ce095ab1347","Type":"ContainerStarted","Data":"f3787b37fc4a2361c87995358f20ec505db3ba63a1e6bcbcf263bc3e9ab11483"}
Oct 30 18:46:20.719668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:20.719627    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9zm7n" event={"ID":"d2f0a477-b0d0-4779-9b17-4ce095ab1347","Type":"ContainerStarted","Data":"b433ef2559e858fd2929ea311c213717fe3c462cc666cfcf5e6b440e1516c2e0"}
Oct 30 18:46:20.720064 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:20.719706    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9zm7n"
Oct 30 18:46:20.827033 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:20.826984    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9zm7n" podStartSLOduration=2.105671665 podStartE2EDuration="8.826970532s" totalImagesPullingTime="6.721298867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:20.825690844 +0000 UTC m=+578.351565940" watchObservedRunningTime="2026-10-30 18:46:20.826970532 +0000 UTC m=+578.352845670"
Oct 30 18:46:26.752651 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:26.752565    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-9zm7n"
Oct 30 18:46:35.529169 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:46:35.529137    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:46:38.787579 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:38.787541    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-577f596567-tj94v"]
Oct 30 18:46:38.805936 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:38.805903    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-tj94v"
Oct 30 18:46:38.858856 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:38.858821    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afefeed1-94b8-448f-9285-6a553e7aa302-webhook-cert\") pod \"maas-controller-577f596567-tj94v\" (UID: \"afefeed1-94b8-448f-9285-6a553e7aa302\") " pod="opendatahub/maas-controller-577f596567-tj94v"
Oct 30 18:46:38.858989 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:38.858902    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afefeed1-94b8-448f-9285-6a553e7aa302-metrics-certs\") pod \"maas-controller-577f596567-tj94v\" (UID: \"afefeed1-94b8-448f-9285-6a553e7aa302\") " pod="opendatahub/maas-controller-577f596567-tj94v"
Oct 30 18:46:38.858989 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:38.858903    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 18:46:38.859105 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:38.859013    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4klww\" (UniqueName: \"kubernetes.io/projected/afefeed1-94b8-448f-9285-6a553e7aa302-kube-api-access-4klww\") pod \"maas-controller-577f596567-tj94v\" (UID: \"afefeed1-94b8-448f-9285-6a553e7aa302\") " pod="opendatahub/maas-controller-577f596567-tj94v"
Oct 30 18:46:38.900744 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:38.900699    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 18:46:38.900884 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:38.900745    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 18:46:38.900884 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:38.900744    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vkhgq\""
Oct 30 18:46:38.901965 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:38.901947    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577f596567-tj94v"]
Oct 30 18:46:38.959925 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:38.959899    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4klww\" (UniqueName: \"kubernetes.io/projected/afefeed1-94b8-448f-9285-6a553e7aa302-kube-api-access-4klww\") pod \"maas-controller-577f596567-tj94v\" (UID: \"afefeed1-94b8-448f-9285-6a553e7aa302\") " pod="opendatahub/maas-controller-577f596567-tj94v"
Oct 30 18:46:38.960052 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:38.959945    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afefeed1-94b8-448f-9285-6a553e7aa302-webhook-cert\") pod \"maas-controller-577f596567-tj94v\" (UID: \"afefeed1-94b8-448f-9285-6a553e7aa302\") " pod="opendatahub/maas-controller-577f596567-tj94v"
Oct 30 18:46:38.960052 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:38.959966    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afefeed1-94b8-448f-9285-6a553e7aa302-metrics-certs\") pod \"maas-controller-577f596567-tj94v\" (UID: \"afefeed1-94b8-448f-9285-6a553e7aa302\") " pod="opendatahub/maas-controller-577f596567-tj94v"
Oct 30 18:46:38.960136 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:46:38.960047    4544 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Oct 30 18:46:38.960136 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:46:38.960112    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afefeed1-94b8-448f-9285-6a553e7aa302-webhook-cert podName:afefeed1-94b8-448f-9285-6a553e7aa302 nodeName:}" failed. No retries permitted until 2026-10-30 18:46:39.460096036 +0000 UTC m=+596.985971115 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/afefeed1-94b8-448f-9285-6a553e7aa302-webhook-cert") pod "maas-controller-577f596567-tj94v" (UID: "afefeed1-94b8-448f-9285-6a553e7aa302") : secret "maas-controller-webhook-cert" not found
Oct 30 18:46:38.962245 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:38.962221    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afefeed1-94b8-448f-9285-6a553e7aa302-metrics-certs\") pod \"maas-controller-577f596567-tj94v\" (UID: \"afefeed1-94b8-448f-9285-6a553e7aa302\") " pod="opendatahub/maas-controller-577f596567-tj94v"
Oct 30 18:46:39.016978 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:39.016953    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4klww\" (UniqueName: \"kubernetes.io/projected/afefeed1-94b8-448f-9285-6a553e7aa302-kube-api-access-4klww\") pod \"maas-controller-577f596567-tj94v\" (UID: \"afefeed1-94b8-448f-9285-6a553e7aa302\") " pod="opendatahub/maas-controller-577f596567-tj94v"
Oct 30 18:46:39.306094 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:39.306067    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7496565bcf-dfskg"]
Oct 30 18:46:39.309477 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:39.309459    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7496565bcf-dfskg"
Oct 30 18:46:39.363359 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:39.363330    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/659356cf-1afb-42ba-8d0e-d489739f4810-metrics-certs\") pod \"maas-controller-7496565bcf-dfskg\" (UID: \"659356cf-1afb-42ba-8d0e-d489739f4810\") " pod="opendatahub/maas-controller-7496565bcf-dfskg"
Oct 30 18:46:39.363493 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:39.363374    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/659356cf-1afb-42ba-8d0e-d489739f4810-webhook-cert\") pod \"maas-controller-7496565bcf-dfskg\" (UID: \"659356cf-1afb-42ba-8d0e-d489739f4810\") " pod="opendatahub/maas-controller-7496565bcf-dfskg"
Oct 30 18:46:39.363493 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:39.363423    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvxp4\" (UniqueName: \"kubernetes.io/projected/659356cf-1afb-42ba-8d0e-d489739f4810-kube-api-access-rvxp4\") pod \"maas-controller-7496565bcf-dfskg\" (UID: \"659356cf-1afb-42ba-8d0e-d489739f4810\") " pod="opendatahub/maas-controller-7496565bcf-dfskg"
Oct 30 18:46:39.364834 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:39.364804    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7496565bcf-dfskg"]
Oct 30 18:46:39.464827 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:39.464799    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afefeed1-94b8-448f-9285-6a553e7aa302-webhook-cert\") pod \"maas-controller-577f596567-tj94v\" (UID: \"afefeed1-94b8-448f-9285-6a553e7aa302\") " pod="opendatahub/maas-controller-577f596567-tj94v"
Oct 30 18:46:39.464827 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:39.464830    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/659356cf-1afb-42ba-8d0e-d489739f4810-metrics-certs\") pod \"maas-controller-7496565bcf-dfskg\" (UID: \"659356cf-1afb-42ba-8d0e-d489739f4810\") " pod="opendatahub/maas-controller-7496565bcf-dfskg"
Oct 30 18:46:39.464987 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:39.464848    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/659356cf-1afb-42ba-8d0e-d489739f4810-webhook-cert\") pod \"maas-controller-7496565bcf-dfskg\" (UID: \"659356cf-1afb-42ba-8d0e-d489739f4810\") " pod="opendatahub/maas-controller-7496565bcf-dfskg"
Oct 30 18:46:39.464987 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:39.464884    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvxp4\" (UniqueName: \"kubernetes.io/projected/659356cf-1afb-42ba-8d0e-d489739f4810-kube-api-access-rvxp4\") pod \"maas-controller-7496565bcf-dfskg\" (UID: \"659356cf-1afb-42ba-8d0e-d489739f4810\") " pod="opendatahub/maas-controller-7496565bcf-dfskg"
Oct 30 18:46:39.467161 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:39.467134    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afefeed1-94b8-448f-9285-6a553e7aa302-webhook-cert\") pod \"maas-controller-577f596567-tj94v\" (UID: \"afefeed1-94b8-448f-9285-6a553e7aa302\") " pod="opendatahub/maas-controller-577f596567-tj94v"
Oct 30 18:46:39.467257 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:39.467230    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/659356cf-1afb-42ba-8d0e-d489739f4810-metrics-certs\") pod \"maas-controller-7496565bcf-dfskg\" (UID: \"659356cf-1afb-42ba-8d0e-d489739f4810\") " pod="opendatahub/maas-controller-7496565bcf-dfskg"
Oct 30 18:46:39.467298 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:39.467230    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/659356cf-1afb-42ba-8d0e-d489739f4810-webhook-cert\") pod \"maas-controller-7496565bcf-dfskg\" (UID: \"659356cf-1afb-42ba-8d0e-d489739f4810\") " pod="opendatahub/maas-controller-7496565bcf-dfskg"
Oct 30 18:46:39.518132 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:39.518110    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvxp4\" (UniqueName: \"kubernetes.io/projected/659356cf-1afb-42ba-8d0e-d489739f4810-kube-api-access-rvxp4\") pod \"maas-controller-7496565bcf-dfskg\" (UID: \"659356cf-1afb-42ba-8d0e-d489739f4810\") " pod="opendatahub/maas-controller-7496565bcf-dfskg"
Oct 30 18:46:39.621529 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:39.621470    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7496565bcf-dfskg"
Oct 30 18:46:39.718800 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:39.718766    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-tj94v"
Oct 30 18:46:39.986387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:39.986316    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7496565bcf-dfskg"]
Oct 30 18:46:39.990067 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:46:39.990041    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod659356cf_1afb_42ba_8d0e_d489739f4810.slice/crio-0488063ff5b630db5b403b3628069855b5a5b2d1fb487317892b71e27760ba64 WatchSource:0}: Error finding container 0488063ff5b630db5b403b3628069855b5a5b2d1fb487317892b71e27760ba64: Status 404 returned error can't find the container with id 0488063ff5b630db5b403b3628069855b5a5b2d1fb487317892b71e27760ba64
Oct 30 18:46:40.093256 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:40.093234    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577f596567-tj94v"]
Oct 30 18:46:40.095596 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:46:40.095569    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafefeed1_94b8_448f_9285_6a553e7aa302.slice/crio-ef88d76fbfc96ec11971b0f7945d2fd886788b9679aed64358a7d763dca769f9 WatchSource:0}: Error finding container ef88d76fbfc96ec11971b0f7945d2fd886788b9679aed64358a7d763dca769f9: Status 404 returned error can't find the container with id ef88d76fbfc96ec11971b0f7945d2fd886788b9679aed64358a7d763dca769f9
Oct 30 18:46:40.796323 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:40.796287    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7496565bcf-dfskg" event={"ID":"659356cf-1afb-42ba-8d0e-d489739f4810","Type":"ContainerStarted","Data":"0488063ff5b630db5b403b3628069855b5a5b2d1fb487317892b71e27760ba64"}
Oct 30 18:46:40.797440 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:40.797410    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-tj94v" event={"ID":"afefeed1-94b8-448f-9285-6a553e7aa302","Type":"ContainerStarted","Data":"ef88d76fbfc96ec11971b0f7945d2fd886788b9679aed64358a7d763dca769f9"}
Oct 30 18:46:42.665193 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:42.665162    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 18:46:42.665563 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:42.665381    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 18:46:44.975990 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:44.975965    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 18:46:45.820108 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:45.820063    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7496565bcf-dfskg" event={"ID":"659356cf-1afb-42ba-8d0e-d489739f4810","Type":"ContainerStarted","Data":"f4d70156fbb5069ba50d2b8029af923a68a82fb533b04913ecc67974a4d771a3"}
Oct 30 18:46:45.820308 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:45.820155    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7496565bcf-dfskg"
Oct 30 18:46:45.822033 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:45.822010    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-tj94v" event={"ID":"afefeed1-94b8-448f-9285-6a553e7aa302","Type":"ContainerStarted","Data":"996dd19754c7c3bbc120e2c1db2942da420e7b85d9481affe255b194ffc58826"}
Oct 30 18:46:45.822174 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:45.822142    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-577f596567-tj94v"
Oct 30 18:46:45.940847 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:45.940803    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-7496565bcf-dfskg" podStartSLOduration=1.998726568 podStartE2EDuration="6.940791349s" totalImagesPullingTime="4.942064781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:45.939965976 +0000 UTC m=+603.465841053" watchObservedRunningTime="2026-10-30 18:46:45.940791349 +0000 UTC m=+603.466666445"
Oct 30 18:46:46.053618 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:46.053552    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-577f596567-tj94v" podStartSLOduration=3.224086034 podStartE2EDuration="8.053539055s" totalImagesPullingTime="4.829453021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:46.052539449 +0000 UTC m=+603.578414544" watchObservedRunningTime="2026-10-30 18:46:46.053539055 +0000 UTC m=+603.579414149"
Oct 30 18:46:49.859270 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:49.859239    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"]
Oct 30 18:46:49.863033 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:49.863014    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"
Oct 30 18:46:49.911990 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:49.911964    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Oct 30 18:46:49.911990 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:49.911967    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Oct 30 18:46:49.954713 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:49.954678    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2abf6a89-8238-4417-8f77-8346ca0b7780-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jnwbh\" (UID: \"2abf6a89-8238-4417-8f77-8346ca0b7780\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"
Oct 30 18:46:49.954862 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:49.954745    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqm5z\" (UniqueName: \"kubernetes.io/projected/2abf6a89-8238-4417-8f77-8346ca0b7780-kube-api-access-jqm5z\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jnwbh\" (UID: \"2abf6a89-8238-4417-8f77-8346ca0b7780\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"
Oct 30 18:46:49.954862 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:49.954836    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/2abf6a89-8238-4417-8f77-8346ca0b7780-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jnwbh\" (UID: \"2abf6a89-8238-4417-8f77-8346ca0b7780\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"
Oct 30 18:46:49.954946 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:49.954880    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2abf6a89-8238-4417-8f77-8346ca0b7780-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jnwbh\" (UID: \"2abf6a89-8238-4417-8f77-8346ca0b7780\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"
Oct 30 18:46:49.958175 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:49.958159    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-4kqvk\""
Oct 30 18:46:49.958260 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:49.958158    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Oct 30 18:46:49.959774 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:49.959755    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"]
Oct 30 18:46:50.055471 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:50.055437    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2abf6a89-8238-4417-8f77-8346ca0b7780-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jnwbh\" (UID: \"2abf6a89-8238-4417-8f77-8346ca0b7780\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"
Oct 30 18:46:50.055670 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:50.055496    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2abf6a89-8238-4417-8f77-8346ca0b7780-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jnwbh\" (UID: \"2abf6a89-8238-4417-8f77-8346ca0b7780\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"
Oct 30 18:46:50.055670 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:50.055655    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqm5z\" (UniqueName: \"kubernetes.io/projected/2abf6a89-8238-4417-8f77-8346ca0b7780-kube-api-access-jqm5z\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jnwbh\" (UID: \"2abf6a89-8238-4417-8f77-8346ca0b7780\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"
Oct 30 18:46:50.055794 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:50.055712    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/2abf6a89-8238-4417-8f77-8346ca0b7780-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jnwbh\" (UID: \"2abf6a89-8238-4417-8f77-8346ca0b7780\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"
Oct 30 18:46:50.055858 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:46:50.055809    4544 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found
Oct 30 18:46:50.055911 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:46:50.055878    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2abf6a89-8238-4417-8f77-8346ca0b7780-proxy-source podName:2abf6a89-8238-4417-8f77-8346ca0b7780 nodeName:}" failed. No retries permitted until 2026-10-30 18:46:50.555857659 +0000 UTC m=+608.081732747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/2abf6a89-8238-4417-8f77-8346ca0b7780-proxy-source") pod "usage-logs-tenancy-proxy-7d94f55f4b-jnwbh" (UID: "2abf6a89-8238-4417-8f77-8346ca0b7780") : configmap "usage-tenancy-proxy-source" not found
Oct 30 18:46:50.056092 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:50.056074    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2abf6a89-8238-4417-8f77-8346ca0b7780-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jnwbh\" (UID: \"2abf6a89-8238-4417-8f77-8346ca0b7780\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"
Oct 30 18:46:50.057909 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:50.057888    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2abf6a89-8238-4417-8f77-8346ca0b7780-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jnwbh\" (UID: \"2abf6a89-8238-4417-8f77-8346ca0b7780\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"
Oct 30 18:46:50.110652 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:50.110572    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqm5z\" (UniqueName: \"kubernetes.io/projected/2abf6a89-8238-4417-8f77-8346ca0b7780-kube-api-access-jqm5z\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jnwbh\" (UID: \"2abf6a89-8238-4417-8f77-8346ca0b7780\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"
Oct 30 18:46:50.560041 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:50.560014    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/2abf6a89-8238-4417-8f77-8346ca0b7780-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jnwbh\" (UID: \"2abf6a89-8238-4417-8f77-8346ca0b7780\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"
Oct 30 18:46:50.560759 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:50.560739    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/2abf6a89-8238-4417-8f77-8346ca0b7780-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jnwbh\" (UID: \"2abf6a89-8238-4417-8f77-8346ca0b7780\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"
Oct 30 18:46:50.774397 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:50.774367    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"
Oct 30 18:46:51.144765 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:51.144674    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"]
Oct 30 18:46:51.147818 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:46:51.147790    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2abf6a89_8238_4417_8f77_8346ca0b7780.slice/crio-6d8f2067ff9be18eb48ce82948df9540145ae54d0f7a9c99a6c52458c3360448 WatchSource:0}: Error finding container 6d8f2067ff9be18eb48ce82948df9540145ae54d0f7a9c99a6c52458c3360448: Status 404 returned error can't find the container with id 6d8f2067ff9be18eb48ce82948df9540145ae54d0f7a9c99a6c52458c3360448
Oct 30 18:46:51.846932 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:51.846899    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh" event={"ID":"2abf6a89-8238-4417-8f77-8346ca0b7780","Type":"ContainerStarted","Data":"6d8f2067ff9be18eb48ce82948df9540145ae54d0f7a9c99a6c52458c3360448"}
Oct 30 18:46:56.829443 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:56.829410    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7496565bcf-dfskg"
Oct 30 18:46:56.829913 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:56.829506    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-577f596567-tj94v"
Oct 30 18:46:57.089803 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.089719    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-577f596567-tj94v"]
Oct 30 18:46:57.090015 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.089966    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-577f596567-tj94v" podUID="afefeed1-94b8-448f-9285-6a553e7aa302" containerName="manager" containerID="cri-o://996dd19754c7c3bbc120e2c1db2942da420e7b85d9481affe255b194ffc58826" gracePeriod=30
Oct 30 18:46:57.356675 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.356595    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-tj94v"
Oct 30 18:46:57.518680 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.518636    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/afefeed1-94b8-448f-9285-6a553e7aa302-metrics-certs\" (UniqueName: \"kubernetes.io/secret/afefeed1-94b8-448f-9285-6a553e7aa302-metrics-certs\") pod \"afefeed1-94b8-448f-9285-6a553e7aa302\" (UID: \"afefeed1-94b8-448f-9285-6a553e7aa302\") "
Oct 30 18:46:57.518855 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.518729    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/afefeed1-94b8-448f-9285-6a553e7aa302-webhook-cert\" (UniqueName: \"kubernetes.io/secret/afefeed1-94b8-448f-9285-6a553e7aa302-webhook-cert\") pod \"afefeed1-94b8-448f-9285-6a553e7aa302\" (UID: \"afefeed1-94b8-448f-9285-6a553e7aa302\") "
Oct 30 18:46:57.518855 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.518809    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/afefeed1-94b8-448f-9285-6a553e7aa302-kube-api-access-4klww\" (UniqueName: \"kubernetes.io/projected/afefeed1-94b8-448f-9285-6a553e7aa302-kube-api-access-4klww\") pod \"afefeed1-94b8-448f-9285-6a553e7aa302\" (UID: \"afefeed1-94b8-448f-9285-6a553e7aa302\") "
Oct 30 18:46:57.521764 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.521732    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afefeed1-94b8-448f-9285-6a553e7aa302-metrics-certs" pod "afefeed1-94b8-448f-9285-6a553e7aa302" (UID: "afefeed1-94b8-448f-9285-6a553e7aa302"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:57.522503 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.522471    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afefeed1-94b8-448f-9285-6a553e7aa302-kube-api-access-4klww" pod "afefeed1-94b8-448f-9285-6a553e7aa302" (UID: "afefeed1-94b8-448f-9285-6a553e7aa302"). InnerVolumeSpecName "kube-api-access-4klww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:57.523139 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.523107    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afefeed1-94b8-448f-9285-6a553e7aa302-webhook-cert" pod "afefeed1-94b8-448f-9285-6a553e7aa302" (UID: "afefeed1-94b8-448f-9285-6a553e7aa302"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:57.619741 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.619667    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4klww\" (UniqueName: \"kubernetes.io/projected/afefeed1-94b8-448f-9285-6a553e7aa302-kube-api-access-4klww\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:46:57.619741 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.619696    4544 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afefeed1-94b8-448f-9285-6a553e7aa302-metrics-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:46:57.619741 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.619706    4544 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afefeed1-94b8-448f-9285-6a553e7aa302-webhook-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:46:57.881714 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.881629    4544 generic.go:356] "Generic (PLEG): container finished" podID="afefeed1-94b8-448f-9285-6a553e7aa302" containerID="996dd19754c7c3bbc120e2c1db2942da420e7b85d9481affe255b194ffc58826" exitCode=0
Oct 30 18:46:57.881714 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.881685    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-tj94v"
Oct 30 18:46:57.881714 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.881696    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-tj94v" event={"ID":"afefeed1-94b8-448f-9285-6a553e7aa302","Type":"ContainerDied","Data":"996dd19754c7c3bbc120e2c1db2942da420e7b85d9481affe255b194ffc58826"}
Oct 30 18:46:57.882261 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.881739    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-tj94v" event={"ID":"afefeed1-94b8-448f-9285-6a553e7aa302","Type":"ContainerDied","Data":"ef88d76fbfc96ec11971b0f7945d2fd886788b9679aed64358a7d763dca769f9"}
Oct 30 18:46:57.882261 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.881764    4544 scope.go:122] "RemoveContainer" containerID="996dd19754c7c3bbc120e2c1db2942da420e7b85d9481affe255b194ffc58826"
Oct 30 18:46:57.891690 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.891669    4544 scope.go:122] "RemoveContainer" containerID="996dd19754c7c3bbc120e2c1db2942da420e7b85d9481affe255b194ffc58826"
Oct 30 18:46:57.891989 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:46:57.891969    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"996dd19754c7c3bbc120e2c1db2942da420e7b85d9481affe255b194ffc58826\": container with ID starting with 996dd19754c7c3bbc120e2c1db2942da420e7b85d9481affe255b194ffc58826 not found: ID does not exist" containerID="996dd19754c7c3bbc120e2c1db2942da420e7b85d9481affe255b194ffc58826"
Oct 30 18:46:57.892076 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:57.892002    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"996dd19754c7c3bbc120e2c1db2942da420e7b85d9481affe255b194ffc58826"} err="failed to get container status \"996dd19754c7c3bbc120e2c1db2942da420e7b85d9481affe255b194ffc58826\": rpc error: code = NotFound desc = could not find container \"996dd19754c7c3bbc120e2c1db2942da420e7b85d9481affe255b194ffc58826\": container with ID starting with 996dd19754c7c3bbc120e2c1db2942da420e7b85d9481affe255b194ffc58826 not found: ID does not exist"
Oct 30 18:46:58.036143 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:58.036109    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-577f596567-tj94v"]
Oct 30 18:46:58.039550 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:58.039521    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-577f596567-tj94v"]
Oct 30 18:46:58.601722 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:46:58.601688    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="afefeed1-94b8-448f-9285-6a553e7aa302" path="/var/lib/kubelet/pods/afefeed1-94b8-448f-9285-6a553e7aa302/volumes"
Oct 30 18:47:04.916517 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:04.916479    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh" event={"ID":"2abf6a89-8238-4417-8f77-8346ca0b7780","Type":"ContainerStarted","Data":"eecd0e5c4fe38f74bc67b60bb0a67fb8d92e16f6af6ce609ae921ddd9cf67b04"}
Oct 30 18:47:04.916900 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:04.916641    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"
Oct 30 18:47:05.027153 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:05.027064    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh" podStartSLOduration=2.404582451 podStartE2EDuration="16.027050084s" totalImagesPullingTime="13.622467633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:05.025889235 +0000 UTC m=+622.551764332" watchObservedRunningTime="2026-10-30 18:47:05.027050084 +0000 UTC m=+622.552925207"
Oct 30 18:47:05.529561 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:47:05.529515    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:47:15.926148 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:15.926120    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jnwbh"
Oct 30 18:47:21.837779 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:21.837741    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"]
Oct 30 18:47:21.838694 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:21.838665    4544 cpu_manager.go:414] "containerMap: removing container" podUID="afefeed1-94b8-448f-9285-6a553e7aa302" containerName="manager"
Oct 30 18:47:21.838694 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:21.838685    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="afefeed1-94b8-448f-9285-6a553e7aa302" containerName="manager"
Oct 30 18:47:21.838841 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:21.838745    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="afefeed1-94b8-448f-9285-6a553e7aa302" containerName="manager"
Oct 30 18:47:21.849117 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:21.849093    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"
Oct 30 18:47:21.898164 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:21.898135    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"]
Oct 30 18:47:21.911849 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:21.911826    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c426d1d-63bf-4145-99f0-7e08baef8a9e-openshift-service-ca\") pod \"maas-api-5c48877c75-l22pj\" (UID: \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"
Oct 30 18:47:21.911966 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:21.911861    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c426d1d-63bf-4145-99f0-7e08baef8a9e-maas-api-tls\") pod \"maas-api-5c48877c75-l22pj\" (UID: \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"
Oct 30 18:47:21.911966 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:21.911890    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5xtc\" (UniqueName: \"kubernetes.io/projected/5c426d1d-63bf-4145-99f0-7e08baef8a9e-kube-api-access-j5xtc\") pod \"maas-api-5c48877c75-l22pj\" (UID: \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"
Oct 30 18:47:21.912047 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:21.912028    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c426d1d-63bf-4145-99f0-7e08baef8a9e-metrics-certs\") pod \"maas-api-5c48877c75-l22pj\" (UID: \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"
Oct 30 18:47:21.942880 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:21.942851    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-977zh\""
Oct 30 18:47:21.943625 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:21.943575    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 18:47:21.943625 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:21.943585    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 18:47:22.012860 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:22.012836    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c426d1d-63bf-4145-99f0-7e08baef8a9e-openshift-service-ca\") pod \"maas-api-5c48877c75-l22pj\" (UID: \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"
Oct 30 18:47:22.012964 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:22.012870    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c426d1d-63bf-4145-99f0-7e08baef8a9e-maas-api-tls\") pod \"maas-api-5c48877c75-l22pj\" (UID: \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"
Oct 30 18:47:22.013087 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:22.013067    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5xtc\" (UniqueName: \"kubernetes.io/projected/5c426d1d-63bf-4145-99f0-7e08baef8a9e-kube-api-access-j5xtc\") pod \"maas-api-5c48877c75-l22pj\" (UID: \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"
Oct 30 18:47:22.013184 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:22.013168    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c426d1d-63bf-4145-99f0-7e08baef8a9e-metrics-certs\") pod \"maas-api-5c48877c75-l22pj\" (UID: \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"
Oct 30 18:47:22.013696 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:22.013670    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c426d1d-63bf-4145-99f0-7e08baef8a9e-openshift-service-ca\") pod \"maas-api-5c48877c75-l22pj\" (UID: \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"
Oct 30 18:47:22.015300 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:22.015280    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c426d1d-63bf-4145-99f0-7e08baef8a9e-maas-api-tls\") pod \"maas-api-5c48877c75-l22pj\" (UID: \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"
Oct 30 18:47:22.015377 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:22.015306    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c426d1d-63bf-4145-99f0-7e08baef8a9e-metrics-certs\") pod \"maas-api-5c48877c75-l22pj\" (UID: \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"
Oct 30 18:47:22.067227 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:22.067205    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5xtc\" (UniqueName: \"kubernetes.io/projected/5c426d1d-63bf-4145-99f0-7e08baef8a9e-kube-api-access-j5xtc\") pod \"maas-api-5c48877c75-l22pj\" (UID: \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"
Oct 30 18:47:22.160715 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:22.160648    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"
Oct 30 18:47:22.522571 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:22.522544    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"]
Oct 30 18:47:22.526442 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:47:22.526417    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c426d1d_63bf_4145_99f0_7e08baef8a9e.slice/crio-49ad6305f328d8fc38f8cdd2bd4c7345893097a458dbf9e5f9ee3622a61946b1 WatchSource:0}: Error finding container 49ad6305f328d8fc38f8cdd2bd4c7345893097a458dbf9e5f9ee3622a61946b1: Status 404 returned error can't find the container with id 49ad6305f328d8fc38f8cdd2bd4c7345893097a458dbf9e5f9ee3622a61946b1
Oct 30 18:47:22.527760 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:22.527743    4544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:47:22.988469 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:22.988380    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj" event={"ID":"5c426d1d-63bf-4145-99f0-7e08baef8a9e","Type":"ContainerStarted","Data":"49ad6305f328d8fc38f8cdd2bd4c7345893097a458dbf9e5f9ee3622a61946b1"}
Oct 30 18:47:28.009831 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:28.009796    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj" event={"ID":"5c426d1d-63bf-4145-99f0-7e08baef8a9e","Type":"ContainerStarted","Data":"035ab5891686a08dd1723141c69109df3889ff633369ba5c4ca247cd63fd08f7"}
Oct 30 18:47:28.010373 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:28.009930    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"
Oct 30 18:47:28.118699 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:28.118654    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj" podStartSLOduration=2.689172973 podStartE2EDuration="7.118641218s" totalImagesPullingTime="4.429468245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:47:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:28.118293716 +0000 UTC m=+645.644168812" watchObservedRunningTime="2026-10-30 18:47:28.118641218 +0000 UTC m=+645.644516312"
Oct 30 18:47:34.019075 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:47:34.019051    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"
Oct 30 18:47:35.530581 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:47:35.530544    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:48:05.530516 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:48:05.530436    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:48:35.530728 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:48:35.530691    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:48:36.937586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:36.937548    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"]
Oct 30 18:48:36.940892 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:36.940872    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.023202 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.023171    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5bffeda4-a4da-4657-86c0-ec91bc1abd2b-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn\" (UID: \"5bffeda4-a4da-4657-86c0-ec91bc1abd2b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.023388 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.023214    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5bffeda4-a4da-4657-86c0-ec91bc1abd2b-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn\" (UID: \"5bffeda4-a4da-4657-86c0-ec91bc1abd2b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.023388 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.023246    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skmfs\" (UniqueName: \"kubernetes.io/projected/5bffeda4-a4da-4657-86c0-ec91bc1abd2b-kube-api-access-skmfs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn\" (UID: \"5bffeda4-a4da-4657-86c0-ec91bc1abd2b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.023515 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.023399    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5bffeda4-a4da-4657-86c0-ec91bc1abd2b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn\" (UID: \"5bffeda4-a4da-4657-86c0-ec91bc1abd2b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.023515 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.023429    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bffeda4-a4da-4657-86c0-ec91bc1abd2b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn\" (UID: \"5bffeda4-a4da-4657-86c0-ec91bc1abd2b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.023612 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.023567    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bffeda4-a4da-4657-86c0-ec91bc1abd2b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn\" (UID: \"5bffeda4-a4da-4657-86c0-ec91bc1abd2b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.035816 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.035787    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7nxf2\""
Oct 30 18:48:37.035982 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.035892    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Oct 30 18:48:37.035982 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.035942    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 18:48:37.035982 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.035950    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 18:48:37.037476 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.037458    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"]
Oct 30 18:48:37.124299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.124262    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bffeda4-a4da-4657-86c0-ec91bc1abd2b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn\" (UID: \"5bffeda4-a4da-4657-86c0-ec91bc1abd2b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.124488 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.124312    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5bffeda4-a4da-4657-86c0-ec91bc1abd2b-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn\" (UID: \"5bffeda4-a4da-4657-86c0-ec91bc1abd2b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.124488 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.124336    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5bffeda4-a4da-4657-86c0-ec91bc1abd2b-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn\" (UID: \"5bffeda4-a4da-4657-86c0-ec91bc1abd2b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.124488 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.124367    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skmfs\" (UniqueName: \"kubernetes.io/projected/5bffeda4-a4da-4657-86c0-ec91bc1abd2b-kube-api-access-skmfs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn\" (UID: \"5bffeda4-a4da-4657-86c0-ec91bc1abd2b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.124488 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.124426    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5bffeda4-a4da-4657-86c0-ec91bc1abd2b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn\" (UID: \"5bffeda4-a4da-4657-86c0-ec91bc1abd2b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.124488 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.124451    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bffeda4-a4da-4657-86c0-ec91bc1abd2b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn\" (UID: \"5bffeda4-a4da-4657-86c0-ec91bc1abd2b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.124811 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.124790    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5bffeda4-a4da-4657-86c0-ec91bc1abd2b-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn\" (UID: \"5bffeda4-a4da-4657-86c0-ec91bc1abd2b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.124876 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.124856    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5bffeda4-a4da-4657-86c0-ec91bc1abd2b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn\" (UID: \"5bffeda4-a4da-4657-86c0-ec91bc1abd2b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.124876 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.124867    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bffeda4-a4da-4657-86c0-ec91bc1abd2b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn\" (UID: \"5bffeda4-a4da-4657-86c0-ec91bc1abd2b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.126759 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.126734    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5bffeda4-a4da-4657-86c0-ec91bc1abd2b-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn\" (UID: \"5bffeda4-a4da-4657-86c0-ec91bc1abd2b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.126998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.126982    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bffeda4-a4da-4657-86c0-ec91bc1abd2b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn\" (UID: \"5bffeda4-a4da-4657-86c0-ec91bc1abd2b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.178817 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.178792    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skmfs\" (UniqueName: \"kubernetes.io/projected/5bffeda4-a4da-4657-86c0-ec91bc1abd2b-kube-api-access-skmfs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn\" (UID: \"5bffeda4-a4da-4657-86c0-ec91bc1abd2b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.253504 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.253422    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:37.619258 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:37.619223    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"]
Oct 30 18:48:37.621996 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:48:37.621964    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bffeda4_a4da_4657_86c0_ec91bc1abd2b.slice/crio-63cc896db3ea25c240ec0342ee2c2cc47d362364a891dbbe40cebb8d77fecb21 WatchSource:0}: Error finding container 63cc896db3ea25c240ec0342ee2c2cc47d362364a891dbbe40cebb8d77fecb21: Status 404 returned error can't find the container with id 63cc896db3ea25c240ec0342ee2c2cc47d362364a891dbbe40cebb8d77fecb21
Oct 30 18:48:38.294237 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:38.294196    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn" event={"ID":"5bffeda4-a4da-4657-86c0-ec91bc1abd2b","Type":"ContainerStarted","Data":"63cc896db3ea25c240ec0342ee2c2cc47d362364a891dbbe40cebb8d77fecb21"}
Oct 30 18:48:39.298963 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:39.298929    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn" event={"ID":"5bffeda4-a4da-4657-86c0-ec91bc1abd2b","Type":"ContainerStarted","Data":"0f5dc765a35794af1d7c2522264f638e11e0c7502c1dab317f8fd944d6df4c7e"}
Oct 30 18:48:39.406780 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:39.406731    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn" podStartSLOduration=1.843660297 podStartE2EDuration="3.406718779s" totalImagesPullingTime="1.563058482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:39.405756823 +0000 UTC m=+716.931631920" watchObservedRunningTime="2026-10-30 18:48:39.406718779 +0000 UTC m=+716.932593878"
Oct 30 18:48:40.253864 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:40.253822    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:40.931810 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:40.931774    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"]
Oct 30 18:48:40.935226 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:40.935205    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.035363 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.035327    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Oct 30 18:48:41.037106 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.037081    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"]
Oct 30 18:48:41.057441 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.057410    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b247e514-74ee-4b71-8466-b184215bf490-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4hrz6\" (UID: \"b247e514-74ee-4b71-8466-b184215bf490\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.057593 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.057450    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b247e514-74ee-4b71-8466-b184215bf490-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4hrz6\" (UID: \"b247e514-74ee-4b71-8466-b184215bf490\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.057593 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.057477    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b247e514-74ee-4b71-8466-b184215bf490-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4hrz6\" (UID: \"b247e514-74ee-4b71-8466-b184215bf490\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.057692 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.057642    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72zjr\" (UniqueName: \"kubernetes.io/projected/b247e514-74ee-4b71-8466-b184215bf490-kube-api-access-72zjr\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4hrz6\" (UID: \"b247e514-74ee-4b71-8466-b184215bf490\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.057727 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.057707    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b247e514-74ee-4b71-8466-b184215bf490-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4hrz6\" (UID: \"b247e514-74ee-4b71-8466-b184215bf490\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.057763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.057754    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b247e514-74ee-4b71-8466-b184215bf490-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4hrz6\" (UID: \"b247e514-74ee-4b71-8466-b184215bf490\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.158490 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.158457    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72zjr\" (UniqueName: \"kubernetes.io/projected/b247e514-74ee-4b71-8466-b184215bf490-kube-api-access-72zjr\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4hrz6\" (UID: \"b247e514-74ee-4b71-8466-b184215bf490\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.158676 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.158526    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b247e514-74ee-4b71-8466-b184215bf490-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4hrz6\" (UID: \"b247e514-74ee-4b71-8466-b184215bf490\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.158676 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.158551    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b247e514-74ee-4b71-8466-b184215bf490-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4hrz6\" (UID: \"b247e514-74ee-4b71-8466-b184215bf490\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.158676 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.158576    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b247e514-74ee-4b71-8466-b184215bf490-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4hrz6\" (UID: \"b247e514-74ee-4b71-8466-b184215bf490\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.158676 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.158591    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b247e514-74ee-4b71-8466-b184215bf490-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4hrz6\" (UID: \"b247e514-74ee-4b71-8466-b184215bf490\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.158676 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.158626    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b247e514-74ee-4b71-8466-b184215bf490-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4hrz6\" (UID: \"b247e514-74ee-4b71-8466-b184215bf490\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.159041 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.159014    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b247e514-74ee-4b71-8466-b184215bf490-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4hrz6\" (UID: \"b247e514-74ee-4b71-8466-b184215bf490\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.159104 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.159074    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b247e514-74ee-4b71-8466-b184215bf490-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4hrz6\" (UID: \"b247e514-74ee-4b71-8466-b184215bf490\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.159155 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.159109    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b247e514-74ee-4b71-8466-b184215bf490-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4hrz6\" (UID: \"b247e514-74ee-4b71-8466-b184215bf490\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.160901 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.160881    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b247e514-74ee-4b71-8466-b184215bf490-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4hrz6\" (UID: \"b247e514-74ee-4b71-8466-b184215bf490\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.161172 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.161153    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b247e514-74ee-4b71-8466-b184215bf490-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4hrz6\" (UID: \"b247e514-74ee-4b71-8466-b184215bf490\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.212885 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.212813    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72zjr\" (UniqueName: \"kubernetes.io/projected/b247e514-74ee-4b71-8466-b184215bf490-kube-api-access-72zjr\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-4hrz6\" (UID: \"b247e514-74ee-4b71-8466-b184215bf490\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.247694 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.247651    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:41.610762 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:41.610729    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"]
Oct 30 18:48:41.613793 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:48:41.613762    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb247e514_74ee_4b71_8466_b184215bf490.slice/crio-b8f8d7f4418a740de82d97eccc4fa2bbf05eeda8f5c711c8d43d09db13851e94 WatchSource:0}: Error finding container b8f8d7f4418a740de82d97eccc4fa2bbf05eeda8f5c711c8d43d09db13851e94: Status 404 returned error can't find the container with id b8f8d7f4418a740de82d97eccc4fa2bbf05eeda8f5c711c8d43d09db13851e94
Oct 30 18:48:42.312899 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:42.312868    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6" event={"ID":"b247e514-74ee-4b71-8466-b184215bf490","Type":"ContainerStarted","Data":"874ac39c144ce65f3919fa566b0f3a26075e0d96b3238e8f031faa0baebbaa20"}
Oct 30 18:48:42.313230 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:42.312917    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6" event={"ID":"b247e514-74ee-4b71-8466-b184215bf490","Type":"ContainerStarted","Data":"b8f8d7f4418a740de82d97eccc4fa2bbf05eeda8f5c711c8d43d09db13851e94"}
Oct 30 18:48:42.421388 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:42.421339    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6" podStartSLOduration=1.78977226 podStartE2EDuration="2.421323835s" totalImagesPullingTime="631.551575ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:42.420349486 +0000 UTC m=+719.946224581" watchObservedRunningTime="2026-10-30 18:48:42.421323835 +0000 UTC m=+719.947198931"
Oct 30 18:48:43.247811 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:43.247779    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:47.254465 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:47.254424    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:47.267168 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:47.267142    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:47.346243 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:47.346214    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn"
Oct 30 18:48:51.248530 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:51.248493    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:51.261105 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:51.261081    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:51.363813 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:51.363786    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-4hrz6"
Oct 30 18:48:53.203731 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.203685    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"]
Oct 30 18:48:53.208961 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.208931    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.262911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.262866    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a3bc726-52a3-4679-b0ac-9910abdb651e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc\" (UID: \"7a3bc726-52a3-4679-b0ac-9910abdb651e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.263077 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.262929    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3bc726-52a3-4679-b0ac-9910abdb651e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc\" (UID: \"7a3bc726-52a3-4679-b0ac-9910abdb651e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.263077 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.262959    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a3bc726-52a3-4679-b0ac-9910abdb651e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc\" (UID: \"7a3bc726-52a3-4679-b0ac-9910abdb651e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.263077 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.262986    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk2kc\" (UniqueName: \"kubernetes.io/projected/7a3bc726-52a3-4679-b0ac-9910abdb651e-kube-api-access-wk2kc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc\" (UID: \"7a3bc726-52a3-4679-b0ac-9910abdb651e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.263077 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.263059    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a3bc726-52a3-4679-b0ac-9910abdb651e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc\" (UID: \"7a3bc726-52a3-4679-b0ac-9910abdb651e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.263294 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.263093    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a3bc726-52a3-4679-b0ac-9910abdb651e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc\" (UID: \"7a3bc726-52a3-4679-b0ac-9910abdb651e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.304257 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.304230    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Oct 30 18:48:53.305554 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.305532    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"]
Oct 30 18:48:53.363414 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.363387    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a3bc726-52a3-4679-b0ac-9910abdb651e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc\" (UID: \"7a3bc726-52a3-4679-b0ac-9910abdb651e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.363584 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.363427    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a3bc726-52a3-4679-b0ac-9910abdb651e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc\" (UID: \"7a3bc726-52a3-4679-b0ac-9910abdb651e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.363584 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.363492    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a3bc726-52a3-4679-b0ac-9910abdb651e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc\" (UID: \"7a3bc726-52a3-4679-b0ac-9910abdb651e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.363584 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.363527    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3bc726-52a3-4679-b0ac-9910abdb651e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc\" (UID: \"7a3bc726-52a3-4679-b0ac-9910abdb651e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.363584 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.363562    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a3bc726-52a3-4679-b0ac-9910abdb651e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc\" (UID: \"7a3bc726-52a3-4679-b0ac-9910abdb651e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.363837 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.363586    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk2kc\" (UniqueName: \"kubernetes.io/projected/7a3bc726-52a3-4679-b0ac-9910abdb651e-kube-api-access-wk2kc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc\" (UID: \"7a3bc726-52a3-4679-b0ac-9910abdb651e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.364028 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.364004    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3bc726-52a3-4679-b0ac-9910abdb651e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc\" (UID: \"7a3bc726-52a3-4679-b0ac-9910abdb651e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.364241 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.364212    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a3bc726-52a3-4679-b0ac-9910abdb651e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc\" (UID: \"7a3bc726-52a3-4679-b0ac-9910abdb651e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.364374 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.364353    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a3bc726-52a3-4679-b0ac-9910abdb651e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc\" (UID: \"7a3bc726-52a3-4679-b0ac-9910abdb651e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.366299 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.366260    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a3bc726-52a3-4679-b0ac-9910abdb651e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc\" (UID: \"7a3bc726-52a3-4679-b0ac-9910abdb651e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.366399 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.366338    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a3bc726-52a3-4679-b0ac-9910abdb651e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc\" (UID: \"7a3bc726-52a3-4679-b0ac-9910abdb651e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.419649 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.419621    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk2kc\" (UniqueName: \"kubernetes.io/projected/7a3bc726-52a3-4679-b0ac-9910abdb651e-kube-api-access-wk2kc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc\" (UID: \"7a3bc726-52a3-4679-b0ac-9910abdb651e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.521465 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.521439    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:53.883265 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:53.883236    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"]
Oct 30 18:48:53.885877 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:48:53.885850    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a3bc726_52a3_4679_b0ac_9910abdb651e.slice/crio-adca73b7f07c894c323c272d3a9e7f220f872e2c11a24bda76c7bb85540566d6 WatchSource:0}: Error finding container adca73b7f07c894c323c272d3a9e7f220f872e2c11a24bda76c7bb85540566d6: Status 404 returned error can't find the container with id adca73b7f07c894c323c272d3a9e7f220f872e2c11a24bda76c7bb85540566d6
Oct 30 18:48:54.370062 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:54.370020    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc" event={"ID":"7a3bc726-52a3-4679-b0ac-9910abdb651e","Type":"ContainerStarted","Data":"adca73b7f07c894c323c272d3a9e7f220f872e2c11a24bda76c7bb85540566d6"}
Oct 30 18:48:55.374927 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:55.374892    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc" event={"ID":"7a3bc726-52a3-4679-b0ac-9910abdb651e","Type":"ContainerStarted","Data":"c5c211517e7f8b07b8f1d1fda3cc9cc3ec74a51cb34bfc89de2a8cbedb7b2085"}
Oct 30 18:48:55.485019 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:55.484966    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc" podStartSLOduration=1.862715531 podStartE2EDuration="2.484950957s" totalImagesPullingTime="622.235426ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:55.483736021 +0000 UTC m=+733.009611128" watchObservedRunningTime="2026-10-30 18:48:55.484950957 +0000 UTC m=+733.010826052"
Oct 30 18:48:55.522156 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:55.522128    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:48:58.089835 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.089795    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"]
Oct 30 18:48:58.093347 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.093323    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.096518 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.096493    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.096629 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.096534    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.096629 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.096566    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.096724 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.096682    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.096724 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.096704    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mdnr\" (UniqueName: \"kubernetes.io/projected/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-kube-api-access-8mdnr\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.096797 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.096734    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.096797 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.096780    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.096872 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.096845    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.096872 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.096863    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.146092 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.146054    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-g5j8l\""
Oct 30 18:48:58.192218 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.192188    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 18:48:58.192218 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.192189    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 18:48:58.192218 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.192223    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 18:48:58.197889 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.197861    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.198078 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.197904    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.198078 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.197931    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.198078 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.197962    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.198078 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.198002    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.198078 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.198033    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.198292 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.198176    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.198292 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.198231    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mdnr\" (UniqueName: \"kubernetes.io/projected/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-kube-api-access-8mdnr\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.198292 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.198284    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.198446 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.198421    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.198639 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.198588    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.198753 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.198731    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.198940 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.198921    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.199012 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.198941    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.200508 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.200486    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.200657 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.200636    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.206916 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.206894    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"]
Oct 30 18:48:58.251834 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.251799    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.252654 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.252581    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mdnr\" (UniqueName: \"kubernetes.io/projected/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-kube-api-access-8mdnr\") pod \"maas-default-gateway-openshift-default-75d7c785f-c5dck\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.405557 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.405476    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:48:58.867073 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:58.867030    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"]
Oct 30 18:48:58.869527 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:48:58.869499    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc5bccb4_412b_4142_8cf5_8e4f99c6ef41.slice/crio-96cd700c050539254af8d3c95e4f6a13d9c7f2777152b85c62ed613e50e70ac7 WatchSource:0}: Error finding container 96cd700c050539254af8d3c95e4f6a13d9c7f2777152b85c62ed613e50e70ac7: Status 404 returned error can't find the container with id 96cd700c050539254af8d3c95e4f6a13d9c7f2777152b85c62ed613e50e70ac7
Oct 30 18:48:59.393942 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:48:59.393904    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck" event={"ID":"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41","Type":"ContainerStarted","Data":"96cd700c050539254af8d3c95e4f6a13d9c7f2777152b85c62ed613e50e70ac7"}
Oct 30 18:49:03.522572 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:03.522530    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:49:03.534882 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:03.534852    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:49:04.431894 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:04.431859    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc"
Oct 30 18:49:05.070780 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:05.070751    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:49:05.071048 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:05.070815    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:49:05.071048 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:05.070846    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:49:05.426142 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:05.426043    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck" event={"ID":"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41","Type":"ContainerStarted","Data":"dec02520188a5d332a315216dbb185b0c4ebca740bc95bc38ed45c6cebd15e79"}
Oct 30 18:49:05.539365 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:49:05.539324    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:49:05.582879 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:05.582811    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck" podStartSLOduration=1.384520435 podStartE2EDuration="7.582794044s" totalImagesPullingTime="6.198273609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:49:05.582150779 +0000 UTC m=+743.108025876" watchObservedRunningTime="2026-10-30 18:49:05.582794044 +0000 UTC m=+743.108669140"
Oct 30 18:49:06.406220 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:06.406189    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:49:06.407305 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:06.407282    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-c5dck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 18:49:06.407362 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:06.407340    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck" podUID="bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 18:49:07.406613 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:07.406578    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-c5dck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 18:49:07.407002 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:07.406647    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck" podUID="bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 18:49:08.405911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:08.405873    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-c5dck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 18:49:08.406096 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:08.405942    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck" podUID="bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 18:49:09.406353 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:09.406318    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-c5dck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 18:49:09.406852 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:09.406372    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck" podUID="bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 18:49:10.406529 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:10.406486    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-c5dck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 18:49:10.406968 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:10.406543    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck" podUID="bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 18:49:11.406120 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:11.406086    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-c5dck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 18:49:11.406325 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:11.406142    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck" podUID="bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 18:49:12.406061 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:12.406026    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-c5dck container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 30 18:49:12.406466 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:12.406091    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck" podUID="bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 30 18:49:13.204978 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.204938    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"]
Oct 30 18:49:13.220980 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.220937    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.250964 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.250920    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.251123 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.250986    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.251123 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.251065    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.251123 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.251094    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.251245 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.251126    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.251245 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.251148    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d2dn\" (UniqueName: \"kubernetes.io/projected/7bfbe33e-bf31-44f9-95ec-812a39171e94-kube-api-access-7d2dn\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.251311 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.251256    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.251359 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.251341    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.251393 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.251376    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bfbe33e-bf31-44f9-95ec-812a39171e94-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.252014 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.251990    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"]
Oct 30 18:49:13.256494 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.256472    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.266241 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.266216    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"]
Oct 30 18:49:13.311812 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.311774    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"]
Oct 30 18:49:13.352780 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.352747    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.352967 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.352786    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bfbe33e-bf31-44f9-95ec-812a39171e94-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.352967 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.352825    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.352967 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.352854    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.352967 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.352875    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt8pv\" (UniqueName: \"kubernetes.io/projected/2aaac2d4-38ca-46d5-95ea-52356e01647f-kube-api-access-kt8pv\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.353182 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.353061    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.353182 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.353100    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.353182 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.353149    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.353182 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.353171    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.353388 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.353186    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.353388 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.353336    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.353388 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.353378    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.353544 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.353414    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.353544 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.353446    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d2dn\" (UniqueName: \"kubernetes.io/projected/7bfbe33e-bf31-44f9-95ec-812a39171e94-kube-api-access-7d2dn\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.353544 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.353482    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.353544 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.353518    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.353544 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.353528    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.353819 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.353569    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bfbe33e-bf31-44f9-95ec-812a39171e94-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.353819 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.353581    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.353819 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.353644    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2aaac2d4-38ca-46d5-95ea-52356e01647f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.353819 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.353677    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.354275 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.354255    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.355708 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.354296    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.356433 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.356408    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.357690 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.357667    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.405701 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.405667    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:49:13.408310 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.408277    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d2dn\" (UniqueName: \"kubernetes.io/projected/7bfbe33e-bf31-44f9-95ec-812a39171e94-kube-api-access-7d2dn\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.408665 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.408344    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-lwkt6\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.410241 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.410221    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:49:13.454532 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.454494    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.454738 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.454535    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt8pv\" (UniqueName: \"kubernetes.io/projected/2aaac2d4-38ca-46d5-95ea-52356e01647f-kube-api-access-kt8pv\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.454738 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.454575    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.454738 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.454650    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.454914 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.454840    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.454914 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.454906    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.455052 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.454971    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2aaac2d4-38ca-46d5-95ea-52356e01647f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.455052 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.454979    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.455052 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.455002    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.455052 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.455031    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.455255 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.455075    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.455312 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.455257    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.455542 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.455510    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.455741 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.455715    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2aaac2d4-38ca-46d5-95ea-52356e01647f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.457381 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.457356    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.457525 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.457505    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.459920 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.459901    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:49:13.508030 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.508006    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.508137 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.508077    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt8pv\" (UniqueName: \"kubernetes.io/projected/2aaac2d4-38ca-46d5-95ea-52356e01647f-kube-api-access-kt8pv\") pod \"maas-default-gateway-openshift-default-75d7c785f-fxssw\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.534252 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.534235    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:13.574960 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.574917    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:13.952956 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.952932    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"]
Oct 30 18:49:13.955101 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:49:13.955073    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bfbe33e_bf31_44f9_95ec_812a39171e94.slice/crio-3f1d5ee36a3f381838296c59dae84d20436dc69a5dff786997a849cccffb03db WatchSource:0}: Error finding container 3f1d5ee36a3f381838296c59dae84d20436dc69a5dff786997a849cccffb03db: Status 404 returned error can't find the container with id 3f1d5ee36a3f381838296c59dae84d20436dc69a5dff786997a849cccffb03db
Oct 30 18:49:13.957268 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.957240    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:49:13.957334 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.957300    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:49:13.957334 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.957326    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:49:13.963227 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.963204    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"]
Oct 30 18:49:13.967424 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:49:13.967399    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aaac2d4_38ca_46d5_95ea_52356e01647f.slice/crio-811d0acb2854f1d285c4e87de544af0b92db978cde7ca19019b5006e8204e2c4 WatchSource:0}: Error finding container 811d0acb2854f1d285c4e87de544af0b92db978cde7ca19019b5006e8204e2c4: Status 404 returned error can't find the container with id 811d0acb2854f1d285c4e87de544af0b92db978cde7ca19019b5006e8204e2c4
Oct 30 18:49:13.969430 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.969401    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:49:13.969519 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.969468    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:49:13.969519 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:13.969500    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:49:14.469457 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:14.468497    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw" event={"ID":"2aaac2d4-38ca-46d5-95ea-52356e01647f","Type":"ContainerStarted","Data":"48634614b12a7446f65c2d0af746b484963bf34fd8daba368a44839525984fe3"}
Oct 30 18:49:14.469457 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:14.468562    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw" event={"ID":"2aaac2d4-38ca-46d5-95ea-52356e01647f","Type":"ContainerStarted","Data":"811d0acb2854f1d285c4e87de544af0b92db978cde7ca19019b5006e8204e2c4"}
Oct 30 18:49:14.473141 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:14.473109    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6" event={"ID":"7bfbe33e-bf31-44f9-95ec-812a39171e94","Type":"ContainerStarted","Data":"d3c299b1d24f4d071186dfb41bd254263a793c9f1f538ddfd2902497bb618679"}
Oct 30 18:49:14.473372 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:14.473351    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6" event={"ID":"7bfbe33e-bf31-44f9-95ec-812a39171e94","Type":"ContainerStarted","Data":"3f1d5ee36a3f381838296c59dae84d20436dc69a5dff786997a849cccffb03db"}
Oct 30 18:49:14.534440 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:14.534413    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:14.575721 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:14.575689    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:14.582064 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:14.582002    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw" podStartSLOduration=1.5819822289999999 podStartE2EDuration="1.581982229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:49:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:49:14.581093845 +0000 UTC m=+752.106968942" watchObservedRunningTime="2026-10-30 18:49:14.581982229 +0000 UTC m=+752.107857331"
Oct 30 18:49:14.693102 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:14.692702    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6" podStartSLOduration=1.692682129 podStartE2EDuration="1.692682129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:49:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:49:14.690090859 +0000 UTC m=+752.215965959" watchObservedRunningTime="2026-10-30 18:49:14.692682129 +0000 UTC m=+752.218557224"
Oct 30 18:49:15.536688 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:15.536663    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-lwkt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 30 18:49:15.537092 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:15.537070    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6" podUID="7bfbe33e-bf31-44f9-95ec-812a39171e94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 30 18:49:15.613630 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:15.608754    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-fxssw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 18:49:15.613630 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:15.608825    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw" podUID="2aaac2d4-38ca-46d5-95ea-52356e01647f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 18:49:16.535483 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:16.535447    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-lwkt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 30 18:49:16.535695 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:16.535497    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6" podUID="7bfbe33e-bf31-44f9-95ec-812a39171e94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 30 18:49:16.577628 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:16.575997    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-fxssw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 18:49:16.577628 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:16.576052    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw" podUID="2aaac2d4-38ca-46d5-95ea-52356e01647f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 18:49:17.215817 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:17.215777    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-697b76b867-qjnxt"]
Oct 30 18:49:17.220199 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:17.220177    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-697b76b867-qjnxt"
Oct 30 18:49:17.269010 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:17.268911    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-697b76b867-qjnxt"]
Oct 30 18:49:17.321891 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:17.321857    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 30 18:49:17.322039 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:17.321981    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 30 18:49:17.322084 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:17.322059    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-9h7n7\""
Oct 30 18:49:17.326620 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:17.326576    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc555285-8fd2-4063-838c-a18d24e3fd7b-metrics-certs\") pod \"ai-gateway-controller-697b76b867-qjnxt\" (UID: \"dc555285-8fd2-4063-838c-a18d24e3fd7b\") " pod="opendatahub/ai-gateway-controller-697b76b867-qjnxt"
Oct 30 18:49:17.326726 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:17.326653    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlv5b\" (UniqueName: \"kubernetes.io/projected/dc555285-8fd2-4063-838c-a18d24e3fd7b-kube-api-access-tlv5b\") pod \"ai-gateway-controller-697b76b867-qjnxt\" (UID: \"dc555285-8fd2-4063-838c-a18d24e3fd7b\") " pod="opendatahub/ai-gateway-controller-697b76b867-qjnxt"
Oct 30 18:49:17.428003 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:17.427965    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc555285-8fd2-4063-838c-a18d24e3fd7b-metrics-certs\") pod \"ai-gateway-controller-697b76b867-qjnxt\" (UID: \"dc555285-8fd2-4063-838c-a18d24e3fd7b\") " pod="opendatahub/ai-gateway-controller-697b76b867-qjnxt"
Oct 30 18:49:17.428189 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:17.428052    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlv5b\" (UniqueName: \"kubernetes.io/projected/dc555285-8fd2-4063-838c-a18d24e3fd7b-kube-api-access-tlv5b\") pod \"ai-gateway-controller-697b76b867-qjnxt\" (UID: \"dc555285-8fd2-4063-838c-a18d24e3fd7b\") " pod="opendatahub/ai-gateway-controller-697b76b867-qjnxt"
Oct 30 18:49:17.430588 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:17.430551    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc555285-8fd2-4063-838c-a18d24e3fd7b-metrics-certs\") pod \"ai-gateway-controller-697b76b867-qjnxt\" (UID: \"dc555285-8fd2-4063-838c-a18d24e3fd7b\") " pod="opendatahub/ai-gateway-controller-697b76b867-qjnxt"
Oct 30 18:49:17.496677 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:17.496455    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlv5b\" (UniqueName: \"kubernetes.io/projected/dc555285-8fd2-4063-838c-a18d24e3fd7b-kube-api-access-tlv5b\") pod \"ai-gateway-controller-697b76b867-qjnxt\" (UID: \"dc555285-8fd2-4063-838c-a18d24e3fd7b\") " pod="opendatahub/ai-gateway-controller-697b76b867-qjnxt"
Oct 30 18:49:17.533586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:17.533560    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-697b76b867-qjnxt"
Oct 30 18:49:17.534703 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:17.534676    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-lwkt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 30 18:49:17.534820 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:17.534724    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6" podUID="7bfbe33e-bf31-44f9-95ec-812a39171e94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 30 18:49:17.579637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:17.576971    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-fxssw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 18:49:17.579637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:17.577034    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw" podUID="2aaac2d4-38ca-46d5-95ea-52356e01647f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 18:49:17.954726 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:17.954703    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-697b76b867-qjnxt"]
Oct 30 18:49:18.521395 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:18.521367    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-697b76b867-qjnxt" event={"ID":"dc555285-8fd2-4063-838c-a18d24e3fd7b","Type":"ContainerStarted","Data":"b520aff04f2afe286f6b8f056c3ece98162c298133885a130d48b3b6cc24a8c7"}
Oct 30 18:49:18.534909 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:18.534883    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-lwkt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 30 18:49:18.535088 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:18.535069    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6" podUID="7bfbe33e-bf31-44f9-95ec-812a39171e94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 30 18:49:18.580624 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:18.580319    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-fxssw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 18:49:18.580624 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:18.580377    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw" podUID="2aaac2d4-38ca-46d5-95ea-52356e01647f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 18:49:19.534716 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:19.534682    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-lwkt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 30 18:49:19.534896 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:19.534735    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6" podUID="7bfbe33e-bf31-44f9-95ec-812a39171e94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 30 18:49:19.576627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:19.576049    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-fxssw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 18:49:19.576627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:19.576122    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw" podUID="2aaac2d4-38ca-46d5-95ea-52356e01647f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 18:49:20.534717 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:20.534679    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-lwkt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 30 18:49:20.535131 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:20.534762    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6" podUID="7bfbe33e-bf31-44f9-95ec-812a39171e94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 30 18:49:20.576363 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:20.576329    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-fxssw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 18:49:20.576484 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:20.576400    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw" podUID="2aaac2d4-38ca-46d5-95ea-52356e01647f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 18:49:21.534571 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:21.534530    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-lwkt6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 30 18:49:21.534788 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:21.534627    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6" podUID="7bfbe33e-bf31-44f9-95ec-812a39171e94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 30 18:49:21.575991 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:21.575956    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-fxssw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 30 18:49:21.576166 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:21.576020    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw" podUID="2aaac2d4-38ca-46d5-95ea-52356e01647f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 30 18:49:22.808161 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:22.807721    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:22.812001 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:22.808387    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:22.812270 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:22.812216    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:49:22.856936 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:22.856904    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:22.857706 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:22.857530    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:22.858408 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:22.858391    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:49:24.554594 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:24.554556    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-697b76b867-qjnxt" event={"ID":"dc555285-8fd2-4063-838c-a18d24e3fd7b","Type":"ContainerStarted","Data":"55412d42655772666090ddd075cab39c3062fb2923f21e8cc90a83a3bc2513c8"}
Oct 30 18:49:24.555094 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:24.554982    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-697b76b867-qjnxt"
Oct 30 18:49:24.662412 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:24.662363    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-697b76b867-qjnxt" podStartSLOduration=1.936680306 podStartE2EDuration="7.662350959s" totalImagesPullingTime="5.725670653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:49:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:49:24.660707433 +0000 UTC m=+762.186582529" watchObservedRunningTime="2026-10-30 18:49:24.662350959 +0000 UTC m=+762.188226054"
Oct 30 18:49:35.530064 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:49:35.530019    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:49:36.565301 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:36.565272    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-697b76b867-qjnxt"
Oct 30 18:49:36.916890 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:36.916811    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-c8f867457-rcrqp"]
Oct 30 18:49:36.923930 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:36.923908    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-c8f867457-rcrqp"
Oct 30 18:49:36.975780 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:36.975752    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-c8f867457-rcrqp"]
Oct 30 18:49:37.050543 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:37.050512    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae711732-bab8-4403-9bf1-c73a98733597-metrics-certs\") pod \"ai-gateway-controller-c8f867457-rcrqp\" (UID: \"ae711732-bab8-4403-9bf1-c73a98733597\") " pod="opendatahub/ai-gateway-controller-c8f867457-rcrqp"
Oct 30 18:49:37.050543 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:37.050543    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kphxr\" (UniqueName: \"kubernetes.io/projected/ae711732-bab8-4403-9bf1-c73a98733597-kube-api-access-kphxr\") pod \"ai-gateway-controller-c8f867457-rcrqp\" (UID: \"ae711732-bab8-4403-9bf1-c73a98733597\") " pod="opendatahub/ai-gateway-controller-c8f867457-rcrqp"
Oct 30 18:49:37.151559 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:37.151520    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae711732-bab8-4403-9bf1-c73a98733597-metrics-certs\") pod \"ai-gateway-controller-c8f867457-rcrqp\" (UID: \"ae711732-bab8-4403-9bf1-c73a98733597\") " pod="opendatahub/ai-gateway-controller-c8f867457-rcrqp"
Oct 30 18:49:37.151559 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:37.151562    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kphxr\" (UniqueName: \"kubernetes.io/projected/ae711732-bab8-4403-9bf1-c73a98733597-kube-api-access-kphxr\") pod \"ai-gateway-controller-c8f867457-rcrqp\" (UID: \"ae711732-bab8-4403-9bf1-c73a98733597\") " pod="opendatahub/ai-gateway-controller-c8f867457-rcrqp"
Oct 30 18:49:37.153893 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:37.153869    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae711732-bab8-4403-9bf1-c73a98733597-metrics-certs\") pod \"ai-gateway-controller-c8f867457-rcrqp\" (UID: \"ae711732-bab8-4403-9bf1-c73a98733597\") " pod="opendatahub/ai-gateway-controller-c8f867457-rcrqp"
Oct 30 18:49:37.208131 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:37.208071    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kphxr\" (UniqueName: \"kubernetes.io/projected/ae711732-bab8-4403-9bf1-c73a98733597-kube-api-access-kphxr\") pod \"ai-gateway-controller-c8f867457-rcrqp\" (UID: \"ae711732-bab8-4403-9bf1-c73a98733597\") " pod="opendatahub/ai-gateway-controller-c8f867457-rcrqp"
Oct 30 18:49:37.237704 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:37.237680    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-c8f867457-rcrqp"
Oct 30 18:49:37.599115 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:37.599084    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-c8f867457-rcrqp"]
Oct 30 18:49:37.602297 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:49:37.602271    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae711732_bab8_4403_9bf1_c73a98733597.slice/crio-b4a011f00e69ef806fd5588a174f177d6b8900d30f4367ca6fb4230ad862a3b6 WatchSource:0}: Error finding container b4a011f00e69ef806fd5588a174f177d6b8900d30f4367ca6fb4230ad862a3b6: Status 404 returned error can't find the container with id b4a011f00e69ef806fd5588a174f177d6b8900d30f4367ca6fb4230ad862a3b6
Oct 30 18:49:37.622311 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:37.622286    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-c8f867457-rcrqp" event={"ID":"ae711732-bab8-4403-9bf1-c73a98733597","Type":"ContainerStarted","Data":"b4a011f00e69ef806fd5588a174f177d6b8900d30f4367ca6fb4230ad862a3b6"}
Oct 30 18:49:38.628189 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:38.628155    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-c8f867457-rcrqp" event={"ID":"ae711732-bab8-4403-9bf1-c73a98733597","Type":"ContainerStarted","Data":"9bbffd0542e536f498e8422cdd05e4585c1bd0d3f42f1f8c76d46b3d8d40d4e2"}
Oct 30 18:49:38.628549 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:38.628226    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-c8f867457-rcrqp"
Oct 30 18:49:38.735725 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:38.735623    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-c8f867457-rcrqp" podStartSLOduration=1.952481176 podStartE2EDuration="2.735592255s" totalImagesPullingTime="783.111079ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:49:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:49:38.733900241 +0000 UTC m=+776.259775336" watchObservedRunningTime="2026-10-30 18:49:38.735592255 +0000 UTC m=+776.261467345"
Oct 30 18:49:49.635386 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:49.635353    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-c8f867457-rcrqp"
Oct 30 18:49:49.789619 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:49.789573    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-697b76b867-qjnxt"]
Oct 30 18:49:49.789849 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:49.789812    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-697b76b867-qjnxt" podUID="dc555285-8fd2-4063-838c-a18d24e3fd7b" containerName="manager" containerID="cri-o://55412d42655772666090ddd075cab39c3062fb2923f21e8cc90a83a3bc2513c8" gracePeriod=30
Oct 30 18:49:50.035500 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:50.035475    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-697b76b867-qjnxt"
Oct 30 18:49:50.066812 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:50.066788    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc555285-8fd2-4063-838c-a18d24e3fd7b-kube-api-access-tlv5b\" (UniqueName: \"kubernetes.io/projected/dc555285-8fd2-4063-838c-a18d24e3fd7b-kube-api-access-tlv5b\") pod \"dc555285-8fd2-4063-838c-a18d24e3fd7b\" (UID: \"dc555285-8fd2-4063-838c-a18d24e3fd7b\") "
Oct 30 18:49:50.066897 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:50.066819    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc555285-8fd2-4063-838c-a18d24e3fd7b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc555285-8fd2-4063-838c-a18d24e3fd7b-metrics-certs\") pod \"dc555285-8fd2-4063-838c-a18d24e3fd7b\" (UID: \"dc555285-8fd2-4063-838c-a18d24e3fd7b\") "
Oct 30 18:49:50.068800 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:50.068772    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc555285-8fd2-4063-838c-a18d24e3fd7b-kube-api-access-tlv5b" pod "dc555285-8fd2-4063-838c-a18d24e3fd7b" (UID: "dc555285-8fd2-4063-838c-a18d24e3fd7b"). InnerVolumeSpecName "kube-api-access-tlv5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:49:50.068890 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:50.068826    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc555285-8fd2-4063-838c-a18d24e3fd7b-metrics-certs" pod "dc555285-8fd2-4063-838c-a18d24e3fd7b" (UID: "dc555285-8fd2-4063-838c-a18d24e3fd7b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:49:50.167988 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:50.167905    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlv5b\" (UniqueName: \"kubernetes.io/projected/dc555285-8fd2-4063-838c-a18d24e3fd7b-kube-api-access-tlv5b\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:49:50.167988 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:50.167930    4544 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc555285-8fd2-4063-838c-a18d24e3fd7b-metrics-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:49:50.679786 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:50.679752    4544 generic.go:356] "Generic (PLEG): container finished" podID="dc555285-8fd2-4063-838c-a18d24e3fd7b" containerID="55412d42655772666090ddd075cab39c3062fb2923f21e8cc90a83a3bc2513c8" exitCode=0
Oct 30 18:49:50.680223 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:50.679796    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-697b76b867-qjnxt"
Oct 30 18:49:50.680223 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:50.679795    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-697b76b867-qjnxt" event={"ID":"dc555285-8fd2-4063-838c-a18d24e3fd7b","Type":"ContainerDied","Data":"55412d42655772666090ddd075cab39c3062fb2923f21e8cc90a83a3bc2513c8"}
Oct 30 18:49:50.680223 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:50.679919    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-697b76b867-qjnxt" event={"ID":"dc555285-8fd2-4063-838c-a18d24e3fd7b","Type":"ContainerDied","Data":"b520aff04f2afe286f6b8f056c3ece98162c298133885a130d48b3b6cc24a8c7"}
Oct 30 18:49:50.680223 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:50.679943    4544 scope.go:122] "RemoveContainer" containerID="55412d42655772666090ddd075cab39c3062fb2923f21e8cc90a83a3bc2513c8"
Oct 30 18:49:50.687784 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:50.687766    4544 scope.go:122] "RemoveContainer" containerID="55412d42655772666090ddd075cab39c3062fb2923f21e8cc90a83a3bc2513c8"
Oct 30 18:49:50.688072 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:49:50.688051    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55412d42655772666090ddd075cab39c3062fb2923f21e8cc90a83a3bc2513c8\": container with ID starting with 55412d42655772666090ddd075cab39c3062fb2923f21e8cc90a83a3bc2513c8 not found: ID does not exist" containerID="55412d42655772666090ddd075cab39c3062fb2923f21e8cc90a83a3bc2513c8"
Oct 30 18:49:50.688123 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:50.688082    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55412d42655772666090ddd075cab39c3062fb2923f21e8cc90a83a3bc2513c8"} err="failed to get container status \"55412d42655772666090ddd075cab39c3062fb2923f21e8cc90a83a3bc2513c8\": rpc error: code = NotFound desc = could not find container \"55412d42655772666090ddd075cab39c3062fb2923f21e8cc90a83a3bc2513c8\": container with ID starting with 55412d42655772666090ddd075cab39c3062fb2923f21e8cc90a83a3bc2513c8 not found: ID does not exist"
Oct 30 18:49:50.833407 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:50.833380    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-697b76b867-qjnxt"]
Oct 30 18:49:50.879162 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:50.879100    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-697b76b867-qjnxt"]
Oct 30 18:49:52.601810 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:49:52.601776    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc555285-8fd2-4063-838c-a18d24e3fd7b" path="/var/lib/kubelet/pods/dc555285-8fd2-4063-838c-a18d24e3fd7b/volumes"
Oct 30 18:50:05.530962 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:50:05.530922    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:50:22.292695 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.292660    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"]
Oct 30 18:50:22.293152 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.293033    4544 cpu_manager.go:414] "containerMap: removing container" podUID="dc555285-8fd2-4063-838c-a18d24e3fd7b" containerName="manager"
Oct 30 18:50:22.293152 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.293043    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc555285-8fd2-4063-838c-a18d24e3fd7b" containerName="manager"
Oct 30 18:50:22.293152 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.293099    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc555285-8fd2-4063-838c-a18d24e3fd7b" containerName="manager"
Oct 30 18:50:22.297483 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.297462    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.354932 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.354904    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-5f80ac-openshift-default-dockercfg-5qbv4\""
Oct 30 18:50:22.394194 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.394163    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"]
Oct 30 18:50:22.468589 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.468552    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-workload-certs\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.468589 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.468588    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/599ac606-1d53-445d-a863-9fd9564cc549-istio-podinfo\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.468794 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.468682    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/599ac606-1d53-445d-a863-9fd9564cc549-istiod-ca-cert\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.468794 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.468751    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/599ac606-1d53-445d-a863-9fd9564cc549-istio-token\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.468794 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.468787    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-workload-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.468893 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.468802    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-credential-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.468893 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.468820    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-istio-data\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.468972 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.468952    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-istio-envoy\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.469043 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.469031    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hj9x\" (UniqueName: \"kubernetes.io/projected/599ac606-1d53-445d-a863-9fd9564cc549-kube-api-access-6hj9x\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.569972 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.569900    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-workload-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.569972 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.569928    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-credential-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.569972 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.569948    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-istio-data\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.570262 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.569993    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-istio-envoy\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.570262 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.570059    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hj9x\" (UniqueName: \"kubernetes.io/projected/599ac606-1d53-445d-a863-9fd9564cc549-kube-api-access-6hj9x\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.570262 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.570090    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-workload-certs\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.570262 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.570109    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/599ac606-1d53-445d-a863-9fd9564cc549-istio-podinfo\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.570262 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.570139    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/599ac606-1d53-445d-a863-9fd9564cc549-istiod-ca-cert\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.570262 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.570184    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/599ac606-1d53-445d-a863-9fd9564cc549-istio-token\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.570558 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.570432    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-credential-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.570558 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.570539    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-istio-data\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.570687 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.570563    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-workload-certs\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.570812 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.570711    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-workload-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.570904 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.570887    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/599ac606-1d53-445d-a863-9fd9564cc549-istiod-ca-cert\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.572458 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.572439    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-istio-envoy\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.572699 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.572680    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/599ac606-1d53-445d-a863-9fd9564cc549-istio-podinfo\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.629342 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.629303    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/599ac606-1d53-445d-a863-9fd9564cc549-istio-token\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.630859 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.630836    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hj9x\" (UniqueName: \"kubernetes.io/projected/599ac606-1d53-445d-a863-9fd9564cc549-kube-api-access-6hj9x\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.910212 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.910130    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:22.954654 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.954621    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"]
Oct 30 18:50:22.958847 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:22.958814    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.011456 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.011417    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-ec8dab-openshift-default-dockercfg-g5szs\""
Oct 30 18:50:23.020680 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.020587    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"]
Oct 30 18:50:23.077116 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.077077    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-workload-certs\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.077309 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.077183    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-token\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.077309 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.077252    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srfn4\" (UniqueName: \"kubernetes.io/projected/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-kube-api-access-srfn4\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.077309 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.077273    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-podinfo\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.077646 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.077482    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-credential-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.077646 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.077506    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-envoy\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.077646 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.077524    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-data\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.077646 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.077549    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istiod-ca-cert\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.077831 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.077692    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-workload-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.178809 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.178732    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srfn4\" (UniqueName: \"kubernetes.io/projected/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-kube-api-access-srfn4\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.178809 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.178776    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-podinfo\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.179033 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.178937    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-credential-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.179033 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.178969    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-envoy\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.179033 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.178984    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-data\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.179033 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.179015    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istiod-ca-cert\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.179244 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.179055    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-workload-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.179244 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.179136    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-workload-certs\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.179244 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.179181    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-token\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.179407 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.179334    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-credential-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.179407 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.179362    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-data\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.179515 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.179424    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-workload-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.179655 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.179630    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-workload-certs\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.179799 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.179686    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istiod-ca-cert\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.181179 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.181154    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-envoy\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.181321 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.181305    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-podinfo\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.232999 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.232971    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-token\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.233163 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.233111    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srfn4\" (UniqueName: \"kubernetes.io/projected/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-kube-api-access-srfn4\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.283586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.283555    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:23.333280 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.333246    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"]
Oct 30 18:50:23.347468 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:50:23.347434    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod599ac606_1d53_445d_a863_9fd9564cc549.slice/crio-0f14a946976e33aa66529bc1c9a1328b8bd4549a146d6a221f8e5d0c9ba6e56b WatchSource:0}: Error finding container 0f14a946976e33aa66529bc1c9a1328b8bd4549a146d6a221f8e5d0c9ba6e56b: Status 404 returned error can't find the container with id 0f14a946976e33aa66529bc1c9a1328b8bd4549a146d6a221f8e5d0c9ba6e56b
Oct 30 18:50:23.351649 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.351596    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:50:23.351763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.351698    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:50:23.351829 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.351771    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:50:23.633018 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.632987    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"]
Oct 30 18:50:23.636664 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.636644    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.684286 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.683650    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-podinfo\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.684286 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.683716    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-credential-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.684286 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.683752    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8gq7\" (UniqueName: \"kubernetes.io/projected/be77624e-eeb7-44f4-8d9e-dfd58897e555-kube-api-access-s8gq7\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.684286 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.683777    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-workload-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.684286 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.683813    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-envoy\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.684286 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.683854    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-token\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.684286 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.683935    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be77624e-eeb7-44f4-8d9e-dfd58897e555-istiod-ca-cert\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.684286 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.683974    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-data\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.684286 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.684002    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-workload-certs\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.695635 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.695592    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"]
Oct 30 18:50:23.696965 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:50:23.696938    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0da27f8b_2d0d_4737_a02f_ab1e1d7d150b.slice/crio-597bdea135667d6db6b2af41eadebab44504ad991fe7aae2978cd02dbcc9a0d1 WatchSource:0}: Error finding container 597bdea135667d6db6b2af41eadebab44504ad991fe7aae2978cd02dbcc9a0d1: Status 404 returned error can't find the container with id 597bdea135667d6db6b2af41eadebab44504ad991fe7aae2978cd02dbcc9a0d1
Oct 30 18:50:23.699919 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.699881    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:50:23.700012 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.699966    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:50:23.700066 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.700009    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:50:23.710237 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.710216    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"]
Oct 30 18:50:23.713523 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.713500    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-75e91b-openshift-default-dockercfg-mcrrx\""
Oct 30 18:50:23.787081 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.786985    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-podinfo\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.787463 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.787301    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-credential-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.787463 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.787378    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8gq7\" (UniqueName: \"kubernetes.io/projected/be77624e-eeb7-44f4-8d9e-dfd58897e555-kube-api-access-s8gq7\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.787463 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.787412    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-workload-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.787965 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.787902    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-credential-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.789549 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.788184    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-envoy\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.789549 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.788245    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-token\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.789549 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.788302    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be77624e-eeb7-44f4-8d9e-dfd58897e555-istiod-ca-cert\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.789549 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.788345    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-data\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.789549 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.788373    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-workload-certs\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.789549 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.788734    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-workload-certs\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.789549 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.789255    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-workload-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.790469 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.790439    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be77624e-eeb7-44f4-8d9e-dfd58897e555-istiod-ca-cert\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.790927 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.790908    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-data\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.800270 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.800014    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-envoy\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.806311 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.800700    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-podinfo\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.828931 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.828079    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk" event={"ID":"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b","Type":"ContainerStarted","Data":"a59670c48701cef89ae80e744176158fafe263bb19e716e4b1a62e151911e04e"}
Oct 30 18:50:23.828931 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.828139    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk" event={"ID":"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b","Type":"ContainerStarted","Data":"597bdea135667d6db6b2af41eadebab44504ad991fe7aae2978cd02dbcc9a0d1"}
Oct 30 18:50:23.831384 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.830797    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df" event={"ID":"599ac606-1d53-445d-a863-9fd9564cc549","Type":"ContainerStarted","Data":"64e54bec5b32ada1c0c7a5d6652d04e4193f6e89a72d0464c3b74e382d3712c8"}
Oct 30 18:50:23.831384 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.830844    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df" event={"ID":"599ac606-1d53-445d-a863-9fd9564cc549","Type":"ContainerStarted","Data":"0f14a946976e33aa66529bc1c9a1328b8bd4549a146d6a221f8e5d0c9ba6e56b"}
Oct 30 18:50:23.844340 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.844316    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8gq7\" (UniqueName: \"kubernetes.io/projected/be77624e-eeb7-44f4-8d9e-dfd58897e555-kube-api-access-s8gq7\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.853222 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.853198    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-token\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.910411 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.910345    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:23.953688 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.953665    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:23.960092 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:23.960049    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk" podStartSLOduration=1.9600358020000002 podStartE2EDuration="1.960035802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:23.959560247 +0000 UTC m=+821.485435341" watchObservedRunningTime="2026-10-30 18:50:23.960035802 +0000 UTC m=+821.485910898"
Oct 30 18:50:24.080478 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:24.080412    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df" podStartSLOduration=2.08039362 podStartE2EDuration="2.08039362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:24.079587733 +0000 UTC m=+821.605462827" watchObservedRunningTime="2026-10-30 18:50:24.08039362 +0000 UTC m=+821.606268718"
Oct 30 18:50:24.284102 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:24.284066    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:24.289114 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:24.289092    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:24.336462 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:24.336437    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"]
Oct 30 18:50:24.339147 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:50:24.339121    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe77624e_eeb7_44f4_8d9e_dfd58897e555.slice/crio-1545326c5c92188d6119166f704fc3e50db15dd2572720ac29944a024a72f64e WatchSource:0}: Error finding container 1545326c5c92188d6119166f704fc3e50db15dd2572720ac29944a024a72f64e: Status 404 returned error can't find the container with id 1545326c5c92188d6119166f704fc3e50db15dd2572720ac29944a024a72f64e
Oct 30 18:50:24.341340 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:24.341315    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:50:24.341401 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:24.341368    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:50:24.341438 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:24.341407    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:50:24.836446 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:24.836415    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75" event={"ID":"be77624e-eeb7-44f4-8d9e-dfd58897e555","Type":"ContainerStarted","Data":"888e1cc8d1cc67f4c3a25002746cd0b478cd3c808ace5d51a6344c4dc8fb8149"}
Oct 30 18:50:24.836630 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:24.836477    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75" event={"ID":"be77624e-eeb7-44f4-8d9e-dfd58897e555","Type":"ContainerStarted","Data":"1545326c5c92188d6119166f704fc3e50db15dd2572720ac29944a024a72f64e"}
Oct 30 18:50:24.836721 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:24.836662    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:24.837672 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:24.837652    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:50:24.915824 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:24.915796    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:24.952577 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:24.952530    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75" podStartSLOduration=1.952517312 podStartE2EDuration="1.952517312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:24.951104724 +0000 UTC m=+822.476979820" watchObservedRunningTime="2026-10-30 18:50:24.952517312 +0000 UTC m=+822.478392408"
Oct 30 18:50:24.954188 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:24.954161    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:25.841371 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:25.841341    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:25.842182 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:25.842162    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:25.959217 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:25.959183    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:26.845112 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:26.845077    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:26.845849 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:26.845832    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:50:31.151281 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:31.151249    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl"]
Oct 30 18:50:31.154891 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:31.154873    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl"
Oct 30 18:50:31.250910 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:31.250271    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-649d16-dockercfg-g2p77\""
Oct 30 18:50:31.250910 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:31.250384    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-649d16\""
Oct 30 18:50:31.256169 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:31.256094    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl"]
Oct 30 18:50:31.269373 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:31.269330    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kmpc\" (UniqueName: \"kubernetes.io/projected/e2f8cdee-cc53-42b9-b70b-d025ef19bcd7-kube-api-access-8kmpc\") pod \"payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl\" (UID: \"e2f8cdee-cc53-42b9-b70b-d025ef19bcd7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl"
Oct 30 18:50:31.269720 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:31.269673    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2f8cdee-cc53-42b9-b70b-d025ef19bcd7-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl\" (UID: \"e2f8cdee-cc53-42b9-b70b-d025ef19bcd7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl"
Oct 30 18:50:31.370948 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:31.370915    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kmpc\" (UniqueName: \"kubernetes.io/projected/e2f8cdee-cc53-42b9-b70b-d025ef19bcd7-kube-api-access-8kmpc\") pod \"payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl\" (UID: \"e2f8cdee-cc53-42b9-b70b-d025ef19bcd7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl"
Oct 30 18:50:31.371130 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:31.371005    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2f8cdee-cc53-42b9-b70b-d025ef19bcd7-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl\" (UID: \"e2f8cdee-cc53-42b9-b70b-d025ef19bcd7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl"
Oct 30 18:50:31.371730 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:31.371710    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2f8cdee-cc53-42b9-b70b-d025ef19bcd7-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl\" (UID: \"e2f8cdee-cc53-42b9-b70b-d025ef19bcd7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl"
Oct 30 18:50:31.426197 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:31.426135    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kmpc\" (UniqueName: \"kubernetes.io/projected/e2f8cdee-cc53-42b9-b70b-d025ef19bcd7-kube-api-access-8kmpc\") pod \"payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl\" (UID: \"e2f8cdee-cc53-42b9-b70b-d025ef19bcd7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl"
Oct 30 18:50:31.468758 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:31.468719    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl"
Oct 30 18:50:31.842245 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:31.842208    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl"]
Oct 30 18:50:31.843317 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:50:31.843286    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2f8cdee_cc53_42b9_b70b_d025ef19bcd7.slice/crio-9e201d8c86a18c99d9372e238b75169b3320994eef47fcbc1ce366f24a82a667 WatchSource:0}: Error finding container 9e201d8c86a18c99d9372e238b75169b3320994eef47fcbc1ce366f24a82a667: Status 404 returned error can't find the container with id 9e201d8c86a18c99d9372e238b75169b3320994eef47fcbc1ce366f24a82a667
Oct 30 18:50:31.868595 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:31.868567    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl" event={"ID":"e2f8cdee-cc53-42b9-b70b-d025ef19bcd7","Type":"ContainerStarted","Data":"9e201d8c86a18c99d9372e238b75169b3320994eef47fcbc1ce366f24a82a667"}
Oct 30 18:50:32.579158 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:32.579123    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"]
Oct 30 18:50:32.583898 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:32.583876    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 18:50:32.651077 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:32.651044    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-5f80ac-serving-cert\""
Oct 30 18:50:32.685394 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:32.685366    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-metrics-certs\") pod \"maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk\" (UID: \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 18:50:32.685549 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:32.685432    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-465p2\" (UniqueName: \"kubernetes.io/projected/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-kube-api-access-465p2\") pod \"maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk\" (UID: \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 18:50:32.685646 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:32.685596    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-maas-api-tls\") pod \"maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk\" (UID: \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 18:50:32.685811 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:32.685784    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk\" (UID: \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 18:50:32.693398 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:32.693376    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"]
Oct 30 18:50:32.787356 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:32.787324    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk\" (UID: \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 18:50:32.787501 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:32.787395    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-metrics-certs\") pod \"maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk\" (UID: \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 18:50:32.787501 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:32.787428    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-465p2\" (UniqueName: \"kubernetes.io/projected/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-kube-api-access-465p2\") pod \"maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk\" (UID: \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 18:50:32.787501 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:32.787463    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-maas-api-tls\") pod \"maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk\" (UID: \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 18:50:32.787685 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:50:32.787554    4544 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-serving-cert: secret "maas-api-e2e-worker-w1-5f80ac-serving-cert" not found
Oct 30 18:50:32.787685 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:50:32.787616    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-maas-api-tls podName:fbd9d51e-133f-4b3a-b592-38ca4b2b5d06 nodeName:}" failed. No retries permitted until 2026-10-30 18:50:33.287590606 +0000 UTC m=+830.813465680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-maas-api-tls") pod "maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk" (UID: "fbd9d51e-133f-4b3a-b592-38ca4b2b5d06") : secret "maas-api-e2e-worker-w1-5f80ac-serving-cert" not found
Oct 30 18:50:32.788124 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:32.788102    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk\" (UID: \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 18:50:32.789827 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:32.789809    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-metrics-certs\") pod \"maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk\" (UID: \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 18:50:32.860700 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:32.848670    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-465p2\" (UniqueName: \"kubernetes.io/projected/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-kube-api-access-465p2\") pod \"maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk\" (UID: \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 18:50:33.293096 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:33.293063    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-maas-api-tls\") pod \"maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk\" (UID: \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 18:50:33.295401 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:33.295373    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-maas-api-tls\") pod \"maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk\" (UID: \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 18:50:33.498124 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:33.498090    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 18:50:33.876224 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:33.876149    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"]
Oct 30 18:50:33.883429 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:50:33.883343    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbd9d51e_133f_4b3a_b592_38ca4b2b5d06.slice/crio-ffe054aca6a4cfb8b577a8e2a2e315b9a8ba23a9956aa9c79a5b765f705ca22f WatchSource:0}: Error finding container ffe054aca6a4cfb8b577a8e2a2e315b9a8ba23a9956aa9c79a5b765f705ca22f: Status 404 returned error can't find the container with id ffe054aca6a4cfb8b577a8e2a2e315b9a8ba23a9956aa9c79a5b765f705ca22f
Oct 30 18:50:34.887741 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:34.887708    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk" event={"ID":"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06","Type":"ContainerStarted","Data":"b7a60343f435000d1f0f372121979f65523d9daab7970d7ae461801bddda83d8"}
Oct 30 18:50:34.888158 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:34.887761    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk" event={"ID":"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06","Type":"ContainerStarted","Data":"ffe054aca6a4cfb8b577a8e2a2e315b9a8ba23a9956aa9c79a5b765f705ca22f"}
Oct 30 18:50:34.888158 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:34.887824    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 18:50:35.039270 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:35.039081    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk" podStartSLOduration=2.145826556 podStartE2EDuration="3.039064416s" totalImagesPullingTime="893.23786ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:35.038510921 +0000 UTC m=+832.564386016" watchObservedRunningTime="2026-10-30 18:50:35.039064416 +0000 UTC m=+832.564939511"
Oct 30 18:50:35.530539 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:50:35.530503    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:50:36.900598 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:36.900544    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl" event={"ID":"e2f8cdee-cc53-42b9-b70b-d025ef19bcd7","Type":"ContainerStarted","Data":"7d3ab6ccf0051e6b5d653906ed40f6e04079a4728008ae8ff860193f37bf4879"}
Oct 30 18:50:36.901092 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:36.900640    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl"
Oct 30 18:50:37.010081 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:37.010023    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl" podStartSLOduration=1.685210962 podStartE2EDuration="6.010008989s" totalImagesPullingTime="4.324798027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:37.008351329 +0000 UTC m=+834.534226426" watchObservedRunningTime="2026-10-30 18:50:37.010008989 +0000 UTC m=+834.535884084"
Oct 30 18:50:37.799795 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:37.797011    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"]
Oct 30 18:50:37.803072 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:37.803035    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 18:50:37.851791 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:37.851759    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-ec8dab-serving-cert\""
Oct 30 18:50:37.857775 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:37.857750    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"]
Oct 30 18:50:37.941680 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:37.941641    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45c27e0-1404-48e3-8289-6426eee88141-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq\" (UID: \"e45c27e0-1404-48e3-8289-6426eee88141\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 18:50:37.942177 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:37.941730    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfg75\" (UniqueName: \"kubernetes.io/projected/e45c27e0-1404-48e3-8289-6426eee88141-kube-api-access-dfg75\") pod \"maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq\" (UID: \"e45c27e0-1404-48e3-8289-6426eee88141\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 18:50:37.942177 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:37.941808    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45c27e0-1404-48e3-8289-6426eee88141-metrics-certs\") pod \"maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq\" (UID: \"e45c27e0-1404-48e3-8289-6426eee88141\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 18:50:37.942177 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:37.941884    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45c27e0-1404-48e3-8289-6426eee88141-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq\" (UID: \"e45c27e0-1404-48e3-8289-6426eee88141\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 18:50:38.042788 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:38.042756    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45c27e0-1404-48e3-8289-6426eee88141-metrics-certs\") pod \"maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq\" (UID: \"e45c27e0-1404-48e3-8289-6426eee88141\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 18:50:38.042949 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:38.042837    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45c27e0-1404-48e3-8289-6426eee88141-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq\" (UID: \"e45c27e0-1404-48e3-8289-6426eee88141\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 18:50:38.042992 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:38.042963    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45c27e0-1404-48e3-8289-6426eee88141-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq\" (UID: \"e45c27e0-1404-48e3-8289-6426eee88141\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 18:50:38.043027 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:50:38.043002    4544 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-serving-cert: secret "maas-api-e2e-worker-w2-ec8dab-serving-cert" not found
Oct 30 18:50:38.043060 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:38.043033    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfg75\" (UniqueName: \"kubernetes.io/projected/e45c27e0-1404-48e3-8289-6426eee88141-kube-api-access-dfg75\") pod \"maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq\" (UID: \"e45c27e0-1404-48e3-8289-6426eee88141\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 18:50:38.043108 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:50:38.043067    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e45c27e0-1404-48e3-8289-6426eee88141-maas-api-tls podName:e45c27e0-1404-48e3-8289-6426eee88141 nodeName:}" failed. No retries permitted until 2026-10-30 18:50:38.543047454 +0000 UTC m=+836.068922548 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e45c27e0-1404-48e3-8289-6426eee88141-maas-api-tls") pod "maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq" (UID: "e45c27e0-1404-48e3-8289-6426eee88141") : secret "maas-api-e2e-worker-w2-ec8dab-serving-cert" not found
Oct 30 18:50:38.043678 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:38.043654    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45c27e0-1404-48e3-8289-6426eee88141-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq\" (UID: \"e45c27e0-1404-48e3-8289-6426eee88141\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 18:50:38.045139 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:38.045121    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45c27e0-1404-48e3-8289-6426eee88141-metrics-certs\") pod \"maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq\" (UID: \"e45c27e0-1404-48e3-8289-6426eee88141\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 18:50:38.098080 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:38.098019    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfg75\" (UniqueName: \"kubernetes.io/projected/e45c27e0-1404-48e3-8289-6426eee88141-kube-api-access-dfg75\") pod \"maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq\" (UID: \"e45c27e0-1404-48e3-8289-6426eee88141\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 18:50:38.559702 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:38.559651    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45c27e0-1404-48e3-8289-6426eee88141-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq\" (UID: \"e45c27e0-1404-48e3-8289-6426eee88141\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 18:50:38.562050 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:38.562018    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45c27e0-1404-48e3-8289-6426eee88141-maas-api-tls\") pod \"maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq\" (UID: \"e45c27e0-1404-48e3-8289-6426eee88141\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 18:50:38.717275 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:38.717243    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 18:50:39.082827 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:39.082803    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"]
Oct 30 18:50:39.085322 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:50:39.085295    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode45c27e0_1404_48e3_8289_6426eee88141.slice/crio-94c5be1c005636ee8d36b807ecb98f3af8589aea005fee474d39cf50c8269cf0 WatchSource:0}: Error finding container 94c5be1c005636ee8d36b807ecb98f3af8589aea005fee474d39cf50c8269cf0: Status 404 returned error can't find the container with id 94c5be1c005636ee8d36b807ecb98f3af8589aea005fee474d39cf50c8269cf0
Oct 30 18:50:39.916654 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:39.916420    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq" event={"ID":"e45c27e0-1404-48e3-8289-6426eee88141","Type":"ContainerStarted","Data":"94c5be1c005636ee8d36b807ecb98f3af8589aea005fee474d39cf50c8269cf0"}
Oct 30 18:50:40.897323 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:40.897294    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 18:50:40.922384 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:40.922348    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq" event={"ID":"e45c27e0-1404-48e3-8289-6426eee88141","Type":"ContainerStarted","Data":"76b0fe820ae60c4adbe3d4b6e0b3f7ed4c70088e4533c97deeb072363c8beef7"}
Oct 30 18:50:40.922519 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:40.922410    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 18:50:41.113782 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:41.113697    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq" podStartSLOduration=3.286508756 podStartE2EDuration="4.113678693s" totalImagesPullingTime="827.169937ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:41.113070528 +0000 UTC m=+838.638945625" watchObservedRunningTime="2026-10-30 18:50:41.113678693 +0000 UTC m=+838.639553789"
Oct 30 18:50:43.084143 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:43.084104    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"]
Oct 30 18:50:43.089651 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:43.089628    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 18:50:43.185655 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:43.185620    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"]
Oct 30 18:50:43.215185 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:43.215159    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh2p2\" (UniqueName: \"kubernetes.io/projected/b7fb9b6d-7af3-4936-9e78-fe761480c080-kube-api-access-qh2p2\") pod \"maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8\" (UID: \"b7fb9b6d-7af3-4936-9e78-fe761480c080\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 18:50:43.215360 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:43.215201    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7fb9b6d-7af3-4936-9e78-fe761480c080-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8\" (UID: \"b7fb9b6d-7af3-4936-9e78-fe761480c080\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 18:50:43.215410 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:43.215354    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7fb9b6d-7af3-4936-9e78-fe761480c080-metrics-certs\") pod \"maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8\" (UID: \"b7fb9b6d-7af3-4936-9e78-fe761480c080\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 18:50:43.215452 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:43.215409    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7fb9b6d-7af3-4936-9e78-fe761480c080-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8\" (UID: \"b7fb9b6d-7af3-4936-9e78-fe761480c080\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 18:50:43.220702 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:43.220682    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-f5705f-serving-cert\""
Oct 30 18:50:43.316667 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:43.316635    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7fb9b6d-7af3-4936-9e78-fe761480c080-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8\" (UID: \"b7fb9b6d-7af3-4936-9e78-fe761480c080\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 18:50:43.316810 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:43.316685    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh2p2\" (UniqueName: \"kubernetes.io/projected/b7fb9b6d-7af3-4936-9e78-fe761480c080-kube-api-access-qh2p2\") pod \"maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8\" (UID: \"b7fb9b6d-7af3-4936-9e78-fe761480c080\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 18:50:43.316810 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:43.316711    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7fb9b6d-7af3-4936-9e78-fe761480c080-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8\" (UID: \"b7fb9b6d-7af3-4936-9e78-fe761480c080\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 18:50:43.316810 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:50:43.316765    4544 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-serving-cert: secret "maas-api-e2e-worker-w0-f5705f-serving-cert" not found
Oct 30 18:50:43.316934 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:50:43.316827    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7fb9b6d-7af3-4936-9e78-fe761480c080-maas-api-tls podName:b7fb9b6d-7af3-4936-9e78-fe761480c080 nodeName:}" failed. No retries permitted until 2026-10-30 18:50:43.816810294 +0000 UTC m=+841.342685388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b7fb9b6d-7af3-4936-9e78-fe761480c080-maas-api-tls") pod "maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8" (UID: "b7fb9b6d-7af3-4936-9e78-fe761480c080") : secret "maas-api-e2e-worker-w0-f5705f-serving-cert" not found
Oct 30 18:50:43.316934 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:43.316845    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7fb9b6d-7af3-4936-9e78-fe761480c080-metrics-certs\") pod \"maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8\" (UID: \"b7fb9b6d-7af3-4936-9e78-fe761480c080\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 18:50:43.317430 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:43.317413    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7fb9b6d-7af3-4936-9e78-fe761480c080-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8\" (UID: \"b7fb9b6d-7af3-4936-9e78-fe761480c080\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 18:50:43.319170 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:43.319140    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7fb9b6d-7af3-4936-9e78-fe761480c080-metrics-certs\") pod \"maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8\" (UID: \"b7fb9b6d-7af3-4936-9e78-fe761480c080\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 18:50:43.388417 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:43.388330    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh2p2\" (UniqueName: \"kubernetes.io/projected/b7fb9b6d-7af3-4936-9e78-fe761480c080-kube-api-access-qh2p2\") pod \"maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8\" (UID: \"b7fb9b6d-7af3-4936-9e78-fe761480c080\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 18:50:43.821476 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:43.821445    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7fb9b6d-7af3-4936-9e78-fe761480c080-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8\" (UID: \"b7fb9b6d-7af3-4936-9e78-fe761480c080\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 18:50:43.823880 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:43.823850    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7fb9b6d-7af3-4936-9e78-fe761480c080-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8\" (UID: \"b7fb9b6d-7af3-4936-9e78-fe761480c080\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 18:50:44.004615 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:44.004576    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 18:50:44.385508 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:44.385480    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"]
Oct 30 18:50:44.388066 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:50:44.388040    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7fb9b6d_7af3_4936_9e78_fe761480c080.slice/crio-0bed570d8fb9a5d991df22b576bbd97368689d31bc2467cd3f285bde8334b063 WatchSource:0}: Error finding container 0bed570d8fb9a5d991df22b576bbd97368689d31bc2467cd3f285bde8334b063: Status 404 returned error can't find the container with id 0bed570d8fb9a5d991df22b576bbd97368689d31bc2467cd3f285bde8334b063
Oct 30 18:50:44.943507 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:44.943471    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8" event={"ID":"b7fb9b6d-7af3-4936-9e78-fe761480c080","Type":"ContainerStarted","Data":"0bed570d8fb9a5d991df22b576bbd97368689d31bc2467cd3f285bde8334b063"}
Oct 30 18:50:45.949756 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:45.949717    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8" event={"ID":"b7fb9b6d-7af3-4936-9e78-fe761480c080","Type":"ContainerStarted","Data":"0501699566a73a921d1af02f88db68cd6dd9de8af12cee4498ab0e8bf17c5bbd"}
Oct 30 18:50:45.950197 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:45.949790    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 18:50:46.934280 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:46.934254    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 18:50:47.048661 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:47.048586    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8" podStartSLOduration=3.189776886 podStartE2EDuration="4.048570137s" totalImagesPullingTime="858.793251ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:46.064202331 +0000 UTC m=+843.590077431" watchObservedRunningTime="2026-10-30 18:50:47.048570137 +0000 UTC m=+844.574445281"
Oct 30 18:50:47.602644 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:47.602593    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8"]
Oct 30 18:50:47.606471 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:47.606452    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8"
Oct 30 18:50:47.657738 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:47.657707    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-3b6ad6-dockercfg-7lflg\""
Oct 30 18:50:47.657738 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:47.657731    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-3b6ad6\""
Oct 30 18:50:47.706023 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:47.705990    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8"]
Oct 30 18:50:47.764849 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:47.764812    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5662c153-5928-418f-b1fa-74bd0d7a2727-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8\" (UID: \"5662c153-5928-418f-b1fa-74bd0d7a2727\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8"
Oct 30 18:50:47.765031 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:47.764874    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lkgd\" (UniqueName: \"kubernetes.io/projected/5662c153-5928-418f-b1fa-74bd0d7a2727-kube-api-access-2lkgd\") pod \"payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8\" (UID: \"5662c153-5928-418f-b1fa-74bd0d7a2727\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8"
Oct 30 18:50:47.865794 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:47.865703    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5662c153-5928-418f-b1fa-74bd0d7a2727-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8\" (UID: \"5662c153-5928-418f-b1fa-74bd0d7a2727\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8"
Oct 30 18:50:47.865794 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:47.865749    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lkgd\" (UniqueName: \"kubernetes.io/projected/5662c153-5928-418f-b1fa-74bd0d7a2727-kube-api-access-2lkgd\") pod \"payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8\" (UID: \"5662c153-5928-418f-b1fa-74bd0d7a2727\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8"
Oct 30 18:50:47.866449 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:47.866418    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5662c153-5928-418f-b1fa-74bd0d7a2727-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8\" (UID: \"5662c153-5928-418f-b1fa-74bd0d7a2727\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8"
Oct 30 18:50:47.909257 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:47.909224    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl"
Oct 30 18:50:47.919794 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:47.919760    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lkgd\" (UniqueName: \"kubernetes.io/projected/5662c153-5928-418f-b1fa-74bd0d7a2727-kube-api-access-2lkgd\") pod \"payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8\" (UID: \"5662c153-5928-418f-b1fa-74bd0d7a2727\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8"
Oct 30 18:50:47.922779 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:47.922556    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8"
Oct 30 18:50:48.294406 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.294380    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8"]
Oct 30 18:50:48.297103 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:50:48.297071    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5662c153_5928_418f_b1fa_74bd0d7a2727.slice/crio-86caf8befac7394734dae9f1bd7ca262864717457069a4942dca97ebfe6342ce WatchSource:0}: Error finding container 86caf8befac7394734dae9f1bd7ca262864717457069a4942dca97ebfe6342ce: Status 404 returned error can't find the container with id 86caf8befac7394734dae9f1bd7ca262864717457069a4942dca97ebfe6342ce
Oct 30 18:50:48.534979 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.534937    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"]
Oct 30 18:50:48.539102 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.539080    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 18:50:48.588393 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.588364    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-3b6ad6-serving-cert\""
Oct 30 18:50:48.621044 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.621012    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"]
Oct 30 18:50:48.674625 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.674561    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dddf9498-52dc-4712-933a-fee14fc5eca9-maas-api-tls\") pod \"maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm\" (UID: \"dddf9498-52dc-4712-933a-fee14fc5eca9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 18:50:48.674625 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.674615    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dddf9498-52dc-4712-933a-fee14fc5eca9-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm\" (UID: \"dddf9498-52dc-4712-933a-fee14fc5eca9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 18:50:48.674867 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.674676    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crzww\" (UniqueName: \"kubernetes.io/projected/dddf9498-52dc-4712-933a-fee14fc5eca9-kube-api-access-crzww\") pod \"maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm\" (UID: \"dddf9498-52dc-4712-933a-fee14fc5eca9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 18:50:48.674867 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.674740    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dddf9498-52dc-4712-933a-fee14fc5eca9-metrics-certs\") pod \"maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm\" (UID: \"dddf9498-52dc-4712-933a-fee14fc5eca9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 18:50:48.776139 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.776097    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dddf9498-52dc-4712-933a-fee14fc5eca9-metrics-certs\") pod \"maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm\" (UID: \"dddf9498-52dc-4712-933a-fee14fc5eca9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 18:50:48.776298 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.776280    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dddf9498-52dc-4712-933a-fee14fc5eca9-maas-api-tls\") pod \"maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm\" (UID: \"dddf9498-52dc-4712-933a-fee14fc5eca9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 18:50:48.776341 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.776312    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dddf9498-52dc-4712-933a-fee14fc5eca9-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm\" (UID: \"dddf9498-52dc-4712-933a-fee14fc5eca9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 18:50:48.776389 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.776361    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crzww\" (UniqueName: \"kubernetes.io/projected/dddf9498-52dc-4712-933a-fee14fc5eca9-kube-api-access-crzww\") pod \"maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm\" (UID: \"dddf9498-52dc-4712-933a-fee14fc5eca9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 18:50:48.776468 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:50:48.776442    4544 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-serving-cert: secret "maas-api-e2e-worker-w5-3b6ad6-serving-cert" not found
Oct 30 18:50:48.776542 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:50:48.776529    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dddf9498-52dc-4712-933a-fee14fc5eca9-maas-api-tls podName:dddf9498-52dc-4712-933a-fee14fc5eca9 nodeName:}" failed. No retries permitted until 2026-10-30 18:50:49.276508982 +0000 UTC m=+846.802384078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dddf9498-52dc-4712-933a-fee14fc5eca9-maas-api-tls") pod "maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm" (UID: "dddf9498-52dc-4712-933a-fee14fc5eca9") : secret "maas-api-e2e-worker-w5-3b6ad6-serving-cert" not found
Oct 30 18:50:48.777027 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.777003    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dddf9498-52dc-4712-933a-fee14fc5eca9-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm\" (UID: \"dddf9498-52dc-4712-933a-fee14fc5eca9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 18:50:48.778533 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.778515    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dddf9498-52dc-4712-933a-fee14fc5eca9-metrics-certs\") pod \"maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm\" (UID: \"dddf9498-52dc-4712-933a-fee14fc5eca9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 18:50:48.832897 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.832831    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crzww\" (UniqueName: \"kubernetes.io/projected/dddf9498-52dc-4712-933a-fee14fc5eca9-kube-api-access-crzww\") pod \"maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm\" (UID: \"dddf9498-52dc-4712-933a-fee14fc5eca9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 18:50:48.987794 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.987759    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8" event={"ID":"5662c153-5928-418f-b1fa-74bd0d7a2727","Type":"ContainerStarted","Data":"31cc546a806f6c86f9a5aeaf3ed4457d8f47aea0bac610f9e8cc458ccec5ff3a"}
Oct 30 18:50:48.987997 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.987821    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8" event={"ID":"5662c153-5928-418f-b1fa-74bd0d7a2727","Type":"ContainerStarted","Data":"86caf8befac7394734dae9f1bd7ca262864717457069a4942dca97ebfe6342ce"}
Oct 30 18:50:48.987997 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:48.987963    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8"
Oct 30 18:50:49.108674 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:49.108073    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8" podStartSLOduration=2.108053497 podStartE2EDuration="2.108053497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:49.100884253 +0000 UTC m=+846.626759348" watchObservedRunningTime="2026-10-30 18:50:49.108053497 +0000 UTC m=+846.633928594"
Oct 30 18:50:49.281325 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:49.281291    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dddf9498-52dc-4712-933a-fee14fc5eca9-maas-api-tls\") pod \"maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm\" (UID: \"dddf9498-52dc-4712-933a-fee14fc5eca9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 18:50:49.283790 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:49.283764    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dddf9498-52dc-4712-933a-fee14fc5eca9-maas-api-tls\") pod \"maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm\" (UID: \"dddf9498-52dc-4712-933a-fee14fc5eca9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 18:50:49.451447 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:49.451361    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 18:50:49.818903 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:49.818832    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"]
Oct 30 18:50:49.821766 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:50:49.821734    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddddf9498_52dc_4712_933a_fee14fc5eca9.slice/crio-b33a001daa29135722a536775653434752518a3e921f9f0ec53fe20fcc5910d9 WatchSource:0}: Error finding container b33a001daa29135722a536775653434752518a3e921f9f0ec53fe20fcc5910d9: Status 404 returned error can't find the container with id b33a001daa29135722a536775653434752518a3e921f9f0ec53fe20fcc5910d9
Oct 30 18:50:49.993773 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:49.993725    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm" event={"ID":"dddf9498-52dc-4712-933a-fee14fc5eca9","Type":"ContainerStarted","Data":"b33a001daa29135722a536775653434752518a3e921f9f0ec53fe20fcc5910d9"}
Oct 30 18:50:51.000583 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:51.000494    4544 generic.go:356] "Generic (PLEG): container finished" podID="599ac606-1d53-445d-a863-9fd9564cc549" containerID="64e54bec5b32ada1c0c7a5d6652d04e4193f6e89a72d0464c3b74e382d3712c8" exitCode=0
Oct 30 18:50:51.001015 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:51.000574    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df" event={"ID":"599ac606-1d53-445d-a863-9fd9564cc549","Type":"ContainerDied","Data":"64e54bec5b32ada1c0c7a5d6652d04e4193f6e89a72d0464c3b74e382d3712c8"}
Oct 30 18:50:51.001083 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:51.001066    4544 scope.go:122] "RemoveContainer" containerID="64e54bec5b32ada1c0c7a5d6652d04e4193f6e89a72d0464c3b74e382d3712c8"
Oct 30 18:50:51.002291 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:51.002266    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm" event={"ID":"dddf9498-52dc-4712-933a-fee14fc5eca9","Type":"ContainerStarted","Data":"341e8e7ca2158530061a3b9127f32999d6ebe40c85f8c4b61c352b14b08cd07c"}
Oct 30 18:50:51.002410 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:51.002391    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 18:50:51.002470 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:51.002433    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:50:51.002511 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:51.002496    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:50:51.002543 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:51.002528    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:50:51.286287 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:51.286229    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm" podStartSLOduration=2.411620732 podStartE2EDuration="3.286209441s" totalImagesPullingTime="874.588709ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:51.284437929 +0000 UTC m=+848.810313028" watchObservedRunningTime="2026-10-30 18:50:51.286209441 +0000 UTC m=+848.812084547"
Oct 30 18:50:52.008225 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:52.006902    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 18:50:52.013385 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:52.013353    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df" event={"ID":"599ac606-1d53-445d-a863-9fd9564cc549","Type":"ContainerStarted","Data":"968e8aa44bc3a01ba9a91a874775e3b73d54b271750d3eddeaeeeb1811eeb4e4"}
Oct 30 18:50:52.912687 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:52.910469    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:52.912687 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:52.911688    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 18:50:52.912687 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:52.911758    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df" podUID="599ac606-1d53-445d-a863-9fd9564cc549" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 18:50:53.911292 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:53.911249    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 18:50:53.911730 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:53.911313    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df" podUID="599ac606-1d53-445d-a863-9fd9564cc549" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 18:50:54.910946 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:54.910907    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 18:50:54.911248 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:54.911215    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df" podUID="599ac606-1d53-445d-a863-9fd9564cc549" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 18:50:55.841314 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:55.841280    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:55.910578 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:55.910541    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 18:50:55.910738 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:55.910599    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df" podUID="599ac606-1d53-445d-a863-9fd9564cc549" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 18:50:56.911651 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:56.911555    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 18:50:56.912167 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:56.911724    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df" podUID="599ac606-1d53-445d-a863-9fd9564cc549" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 18:50:57.022480 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:57.022453    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 18:50:57.915460 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:57.915427    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:57.916669 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:57.916648    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 18:50:58.734879 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:58.734817    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"]
Oct 30 18:50:58.739396 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:58.739373    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 18:50:58.830170 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:58.830139    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15b9134c-0ca4-4285-a041-badc8cddc776-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l\" (UID: \"15b9134c-0ca4-4285-a041-badc8cddc776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 18:50:58.830340 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:58.830186    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15b9134c-0ca4-4285-a041-badc8cddc776-maas-api-tls\") pod \"maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l\" (UID: \"15b9134c-0ca4-4285-a041-badc8cddc776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 18:50:58.830340 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:58.830289    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15b9134c-0ca4-4285-a041-badc8cddc776-metrics-certs\") pod \"maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l\" (UID: \"15b9134c-0ca4-4285-a041-badc8cddc776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 18:50:58.830412 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:58.830344    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hxrl\" (UniqueName: \"kubernetes.io/projected/15b9134c-0ca4-4285-a041-badc8cddc776-kube-api-access-5hxrl\") pod \"maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l\" (UID: \"15b9134c-0ca4-4285-a041-badc8cddc776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 18:50:58.835903 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:58.835880    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-649d16-serving-cert\""
Oct 30 18:50:58.838059 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:58.838041    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"]
Oct 30 18:50:58.931571 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:58.931529    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15b9134c-0ca4-4285-a041-badc8cddc776-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l\" (UID: \"15b9134c-0ca4-4285-a041-badc8cddc776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 18:50:58.932074 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:58.931623    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15b9134c-0ca4-4285-a041-badc8cddc776-maas-api-tls\") pod \"maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l\" (UID: \"15b9134c-0ca4-4285-a041-badc8cddc776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 18:50:58.932074 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:58.931661    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15b9134c-0ca4-4285-a041-badc8cddc776-metrics-certs\") pod \"maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l\" (UID: \"15b9134c-0ca4-4285-a041-badc8cddc776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 18:50:58.932074 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:58.931688    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hxrl\" (UniqueName: \"kubernetes.io/projected/15b9134c-0ca4-4285-a041-badc8cddc776-kube-api-access-5hxrl\") pod \"maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l\" (UID: \"15b9134c-0ca4-4285-a041-badc8cddc776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 18:50:58.932074 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:50:58.931795    4544 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-serving-cert: secret "maas-api-e2e-worker-w6-649d16-serving-cert" not found
Oct 30 18:50:58.932074 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:50:58.931856    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15b9134c-0ca4-4285-a041-badc8cddc776-maas-api-tls podName:15b9134c-0ca4-4285-a041-badc8cddc776 nodeName:}" failed. No retries permitted until 2026-10-30 18:50:59.431841407 +0000 UTC m=+856.957716486 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/15b9134c-0ca4-4285-a041-badc8cddc776-maas-api-tls") pod "maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l" (UID: "15b9134c-0ca4-4285-a041-badc8cddc776") : secret "maas-api-e2e-worker-w6-649d16-serving-cert" not found
Oct 30 18:50:58.932400 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:58.932377    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15b9134c-0ca4-4285-a041-badc8cddc776-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l\" (UID: \"15b9134c-0ca4-4285-a041-badc8cddc776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 18:50:58.934315 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:58.934285    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15b9134c-0ca4-4285-a041-badc8cddc776-metrics-certs\") pod \"maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l\" (UID: \"15b9134c-0ca4-4285-a041-badc8cddc776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 18:50:58.997341 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:58.997263    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hxrl\" (UniqueName: \"kubernetes.io/projected/15b9134c-0ca4-4285-a041-badc8cddc776-kube-api-access-5hxrl\") pod \"maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l\" (UID: \"15b9134c-0ca4-4285-a041-badc8cddc776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 18:50:59.436875 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:59.436844    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15b9134c-0ca4-4285-a041-badc8cddc776-maas-api-tls\") pod \"maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l\" (UID: \"15b9134c-0ca4-4285-a041-badc8cddc776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 18:50:59.439134 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:59.439115    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15b9134c-0ca4-4285-a041-badc8cddc776-maas-api-tls\") pod \"maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l\" (UID: \"15b9134c-0ca4-4285-a041-badc8cddc776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 18:50:59.653031 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:59.652984    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 18:50:59.997293 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:50:59.997227    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8"
Oct 30 18:51:00.028865 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:00.028834    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"]
Oct 30 18:51:00.033006 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:51:00.032972    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15b9134c_0ca4_4285_a041_badc8cddc776.slice/crio-2f5b063cdbc0d9c2cbc55d4bf6ea0487c50ce6055e9e180cc0b0b52d00481487 WatchSource:0}: Error finding container 2f5b063cdbc0d9c2cbc55d4bf6ea0487c50ce6055e9e180cc0b0b52d00481487: Status 404 returned error can't find the container with id 2f5b063cdbc0d9c2cbc55d4bf6ea0487c50ce6055e9e180cc0b0b52d00481487
Oct 30 18:51:00.060860 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:00.060827    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l" event={"ID":"15b9134c-0ca4-4285-a041-badc8cddc776","Type":"ContainerStarted","Data":"2f5b063cdbc0d9c2cbc55d4bf6ea0487c50ce6055e9e180cc0b0b52d00481487"}
Oct 30 18:51:00.062409 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:00.062387    4544 generic.go:356] "Generic (PLEG): container finished" podID="0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" containerID="a59670c48701cef89ae80e744176158fafe263bb19e716e4b1a62e151911e04e" exitCode=0
Oct 30 18:51:00.062525 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:00.062422    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk" event={"ID":"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b","Type":"ContainerDied","Data":"a59670c48701cef89ae80e744176158fafe263bb19e716e4b1a62e151911e04e"}
Oct 30 18:51:00.062778 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:00.062764    4544 scope.go:122] "RemoveContainer" containerID="a59670c48701cef89ae80e744176158fafe263bb19e716e4b1a62e151911e04e"
Oct 30 18:51:00.064032 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:00.064001    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:51:00.064114 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:00.064066    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:51:00.064114 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:00.064096    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:51:01.087312 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:01.087199    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l" event={"ID":"15b9134c-0ca4-4285-a041-badc8cddc776","Type":"ContainerStarted","Data":"2a3d6f377ab168f1bfddad06f2abaab8934c523d1cf1de55a23218b22b3860f1"}
Oct 30 18:51:01.087884 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:01.087858    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 18:51:01.089443 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:01.089423    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk" event={"ID":"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b","Type":"ContainerStarted","Data":"491b582a839a2c50de4f7231ed0bb51e306dc3094feb01f0a8d202c019d67d21"}
Oct 30 18:51:01.198215 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:01.198163    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l" podStartSLOduration=2.426614399 podStartE2EDuration="3.198144557s" totalImagesPullingTime="771.530158ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:01.196823956 +0000 UTC m=+858.722699053" watchObservedRunningTime="2026-10-30 18:51:01.198144557 +0000 UTC m=+858.724019651"
Oct 30 18:51:01.283972 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:01.283918    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:51:01.284188 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:01.284111    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 18:51:01.284188 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:01.284169    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk" podUID="0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 18:51:02.289897 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:02.289850    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 18:51:02.290410 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:02.289940    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk" podUID="0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 18:51:03.284554 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:03.284506    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 18:51:03.285184 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:03.284810    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk" podUID="0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 18:51:04.284153 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:04.284108    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 18:51:04.284549 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:04.284167    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk" podUID="0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 18:51:05.284798 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:05.284744    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 18:51:05.285252 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:05.285229    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk" podUID="0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 18:51:05.530933 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:51:05.530898    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:51:06.288984 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:06.288953    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:51:06.289400 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:06.289358    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:51:06.290169 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:06.290148    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 18:51:07.099715 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:07.099686    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 18:51:24.229192 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:24.229031    4544 generic.go:356] "Generic (PLEG): container finished" podID="be77624e-eeb7-44f4-8d9e-dfd58897e555" containerID="888e1cc8d1cc67f4c3a25002746cd0b478cd3c808ace5d51a6344c4dc8fb8149" exitCode=0
Oct 30 18:51:24.229192 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:24.229078    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75" event={"ID":"be77624e-eeb7-44f4-8d9e-dfd58897e555","Type":"ContainerDied","Data":"888e1cc8d1cc67f4c3a25002746cd0b478cd3c808ace5d51a6344c4dc8fb8149"}
Oct 30 18:51:24.229868 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:24.229507    4544 scope.go:122] "RemoveContainer" containerID="888e1cc8d1cc67f4c3a25002746cd0b478cd3c808ace5d51a6344c4dc8fb8149"
Oct 30 18:51:24.230867 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:24.230835    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:51:24.230970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:24.230911    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:51:24.230970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:24.230946    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:51:25.235362 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:25.235331    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75" event={"ID":"be77624e-eeb7-44f4-8d9e-dfd58897e555","Type":"ContainerStarted","Data":"746ebc4f7c72e36da5fccd7344bb932af66b8a5f37245b10f5aaaca1927d188f"}
Oct 30 18:51:25.954498 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:25.954457    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:51:25.955715 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:25.955687    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 30 18:51:25.955840 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:25.955744    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75" podUID="be77624e-eeb7-44f4-8d9e-dfd58897e555" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 30 18:51:26.845420 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:26.845390    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:51:26.954763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:26.954728    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 30 18:51:26.954930 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:26.954781    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75" podUID="be77624e-eeb7-44f4-8d9e-dfd58897e555" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 30 18:51:27.954886 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:27.954852    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 30 18:51:27.955450 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:27.955419    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75" podUID="be77624e-eeb7-44f4-8d9e-dfd58897e555" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 30 18:51:28.954178 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:28.954142    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 30 18:51:28.954368 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:28.954208    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75" podUID="be77624e-eeb7-44f4-8d9e-dfd58897e555" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 30 18:51:29.955124 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:29.955082    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 30 18:51:29.955624 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:29.955572    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75" podUID="be77624e-eeb7-44f4-8d9e-dfd58897e555" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 30 18:51:30.959248 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:30.959217    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:51:30.960282 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:30.960260    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 18:51:35.530947 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:51:35.530910    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:51:42.697356 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:42.697326    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 18:51:42.700023 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:42.700003    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 18:51:53.199350 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.199318    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"]
Oct 30 18:51:53.202565 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.202545    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.300321 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.300286    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f5705f\"/\"default-dockercfg-d2rht\""
Oct 30 18:51:53.300479 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.300328    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f5705f\"/\"simulator-w0-f5705f-kserve-self-signed-certs\""
Oct 30 18:51:53.300479 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.300377    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-f5705f\"/\"openshift-service-ca.crt\""
Oct 30 18:51:53.300861 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.300845    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-f5705f\"/\"kube-root-ca.crt\""
Oct 30 18:51:53.302339 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.302319    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"]
Oct 30 18:51:53.377654 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.377620    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-model-cache\") pod \"simulator-w0-f5705f-kserve-85499df5bd-7xklc\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") " pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.377654 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.377657    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12f140d3-7cea-4dc9-8a60-e8d11acca08f-tls-certs\") pod \"simulator-w0-f5705f-kserve-85499df5bd-7xklc\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") " pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.377917 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.377675    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-tmp-dir\") pod \"simulator-w0-f5705f-kserve-85499df5bd-7xklc\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") " pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.377917 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.377704    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-home\") pod \"simulator-w0-f5705f-kserve-85499df5bd-7xklc\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") " pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.378021 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.377907    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-dshm\") pod \"simulator-w0-f5705f-kserve-85499df5bd-7xklc\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") " pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.378021 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.377988    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frltf\" (UniqueName: \"kubernetes.io/projected/12f140d3-7cea-4dc9-8a60-e8d11acca08f-kube-api-access-frltf\") pod \"simulator-w0-f5705f-kserve-85499df5bd-7xklc\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") " pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.479176 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.479079    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-model-cache\") pod \"simulator-w0-f5705f-kserve-85499df5bd-7xklc\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") " pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.479176 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.479124    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12f140d3-7cea-4dc9-8a60-e8d11acca08f-tls-certs\") pod \"simulator-w0-f5705f-kserve-85499df5bd-7xklc\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") " pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.479176 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.479167    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-tmp-dir\") pod \"simulator-w0-f5705f-kserve-85499df5bd-7xklc\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") " pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.479451 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.479208    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-home\") pod \"simulator-w0-f5705f-kserve-85499df5bd-7xklc\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") " pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.479451 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.479282    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-dshm\") pod \"simulator-w0-f5705f-kserve-85499df5bd-7xklc\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") " pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.479451 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.479331    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frltf\" (UniqueName: \"kubernetes.io/projected/12f140d3-7cea-4dc9-8a60-e8d11acca08f-kube-api-access-frltf\") pod \"simulator-w0-f5705f-kserve-85499df5bd-7xklc\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") " pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.479627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.479535    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-model-cache\") pod \"simulator-w0-f5705f-kserve-85499df5bd-7xklc\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") " pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.479627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.479555    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-tmp-dir\") pod \"simulator-w0-f5705f-kserve-85499df5bd-7xklc\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") " pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.479899 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.479873    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-home\") pod \"simulator-w0-f5705f-kserve-85499df5bd-7xklc\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") " pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.481527 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.481507    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-dshm\") pod \"simulator-w0-f5705f-kserve-85499df5bd-7xklc\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") " pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.481812 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.481795    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12f140d3-7cea-4dc9-8a60-e8d11acca08f-tls-certs\") pod \"simulator-w0-f5705f-kserve-85499df5bd-7xklc\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") " pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.533519 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.533492    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frltf\" (UniqueName: \"kubernetes.io/projected/12f140d3-7cea-4dc9-8a60-e8d11acca08f-kube-api-access-frltf\") pod \"simulator-w0-f5705f-kserve-85499df5bd-7xklc\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") " pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:53.830177 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:53.830146    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:51:54.196786 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:54.196751    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"]
Oct 30 18:51:54.199033 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:51:54.199009    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12f140d3_7cea_4dc9_8a60_e8d11acca08f.slice/crio-af89b5492fe9fd76137714d568c10deaad4c5b41c7f606e1c6eb338dc71946fb WatchSource:0}: Error finding container af89b5492fe9fd76137714d568c10deaad4c5b41c7f606e1c6eb338dc71946fb: Status 404 returned error can't find the container with id af89b5492fe9fd76137714d568c10deaad4c5b41c7f606e1c6eb338dc71946fb
Oct 30 18:51:54.407638 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:54.407587    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" event={"ID":"12f140d3-7cea-4dc9-8a60-e8d11acca08f","Type":"ContainerStarted","Data":"04a75aa9862defc69ec19016034737f0d3834e74221a54eb91d9a56fbd3f4a1e"}
Oct 30 18:51:54.408034 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:54.407644    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" event={"ID":"12f140d3-7cea-4dc9-8a60-e8d11acca08f","Type":"ContainerStarted","Data":"af89b5492fe9fd76137714d568c10deaad4c5b41c7f606e1c6eb338dc71946fb"}
Oct 30 18:51:54.516792 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:54.516742    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podStartSLOduration=1.5167288349999999 podStartE2EDuration="1.516728835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:54.514877142 +0000 UTC m=+912.040752238" watchObservedRunningTime="2026-10-30 18:51:54.516728835 +0000 UTC m=+912.042603931"
Oct 30 18:51:54.830330 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:51:54.830251    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:52:03.831034 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:03.831000    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:52:03.843856 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:03.843825    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:52:04.471102 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:04.471069    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:52:05.531948 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:52:05.531908    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:52:17.678225 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.678190    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"]
Oct 30 18:52:17.683363 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.683338    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.730108 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.730074    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-tls-certs\") pod \"premium-w1-5f80ac-kserve-58776b59c6-m57l5\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") " pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.730311 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.730127    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-dshm\") pod \"premium-w1-5f80ac-kserve-58776b59c6-m57l5\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") " pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.730311 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.730174    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-tmp-dir\") pod \"premium-w1-5f80ac-kserve-58776b59c6-m57l5\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") " pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.730311 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.730286    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-home\") pod \"premium-w1-5f80ac-kserve-58776b59c6-m57l5\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") " pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.730504 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.730475    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-model-cache\") pod \"premium-w1-5f80ac-kserve-58776b59c6-m57l5\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") " pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.730504 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.730496    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw9bc\" (UniqueName: \"kubernetes.io/projected/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-kube-api-access-cw9bc\") pod \"premium-w1-5f80ac-kserve-58776b59c6-m57l5\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") " pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.757339 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.757296    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"]
Oct 30 18:52:17.781484 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.781449    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-5f80ac\"/\"premium-w1-5f80ac-kserve-self-signed-certs\""
Oct 30 18:52:17.781669 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.781492    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-5f80ac\"/\"default-dockercfg-dt6dr\""
Oct 30 18:52:17.781669 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.781496    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-5f80ac\"/\"openshift-service-ca.crt\""
Oct 30 18:52:17.781669 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.781538    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-5f80ac\"/\"kube-root-ca.crt\""
Oct 30 18:52:17.831594 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.831557    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-tls-certs\") pod \"premium-w1-5f80ac-kserve-58776b59c6-m57l5\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") " pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.831594 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.831620    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-dshm\") pod \"premium-w1-5f80ac-kserve-58776b59c6-m57l5\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") " pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.831863 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.831652    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-tmp-dir\") pod \"premium-w1-5f80ac-kserve-58776b59c6-m57l5\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") " pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.831863 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.831679    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-home\") pod \"premium-w1-5f80ac-kserve-58776b59c6-m57l5\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") " pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.831863 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.831747    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-model-cache\") pod \"premium-w1-5f80ac-kserve-58776b59c6-m57l5\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") " pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.831863 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.831766    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw9bc\" (UniqueName: \"kubernetes.io/projected/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-kube-api-access-cw9bc\") pod \"premium-w1-5f80ac-kserve-58776b59c6-m57l5\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") " pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.832138 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.832114    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-tmp-dir\") pod \"premium-w1-5f80ac-kserve-58776b59c6-m57l5\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") " pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.832190 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.832159    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-home\") pod \"premium-w1-5f80ac-kserve-58776b59c6-m57l5\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") " pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.832226 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.832203    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-model-cache\") pod \"premium-w1-5f80ac-kserve-58776b59c6-m57l5\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") " pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.833922 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.833903    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-dshm\") pod \"premium-w1-5f80ac-kserve-58776b59c6-m57l5\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") " pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.834225 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.834208    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-tls-certs\") pod \"premium-w1-5f80ac-kserve-58776b59c6-m57l5\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") " pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.886559 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.886522    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw9bc\" (UniqueName: \"kubernetes.io/projected/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-kube-api-access-cw9bc\") pod \"premium-w1-5f80ac-kserve-58776b59c6-m57l5\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") " pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:17.997521 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:17.997438    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:18.369678 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:18.369648    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"]
Oct 30 18:52:18.370836 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:52:18.370807    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78d18a17_75d1_49c4_bb28_ab7e2bc5753a.slice/crio-e0bb6cad76e4a17f67882d22ae443aaf1ab5608089af26005d4c4bec33957567 WatchSource:0}: Error finding container e0bb6cad76e4a17f67882d22ae443aaf1ab5608089af26005d4c4bec33957567: Status 404 returned error can't find the container with id e0bb6cad76e4a17f67882d22ae443aaf1ab5608089af26005d4c4bec33957567
Oct 30 18:52:18.532343 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:18.532305    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" event={"ID":"78d18a17-75d1-49c4-bb28-ab7e2bc5753a","Type":"ContainerStarted","Data":"e0b6352e47637d6a07c2d368655a1bda47525b1efb16f79732a58fc66d03fa99"}
Oct 30 18:52:18.532343 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:18.532345    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" event={"ID":"78d18a17-75d1-49c4-bb28-ab7e2bc5753a","Type":"ContainerStarted","Data":"e0bb6cad76e4a17f67882d22ae443aaf1ab5608089af26005d4c4bec33957567"}
Oct 30 18:52:18.641323 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:18.641219    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podStartSLOduration=1.641206071 podStartE2EDuration="1.641206071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:18.639580649 +0000 UTC m=+936.165455745" watchObservedRunningTime="2026-10-30 18:52:18.641206071 +0000 UTC m=+936.167081166"
Oct 30 18:52:18.997968 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:18.997887    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:27.998336 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:27.998131    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:28.011557 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:28.011524    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:28.621597 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:28.621266    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:52:35.530874 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:52:35.530838    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:52:42.168851 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.168815    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"]
Oct 30 18:52:42.172855 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.172833    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.204695 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.204667    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-tmp-dir\") pod \"simulator-w3-a1617e-kserve-8576757679-9k482\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") " pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.204824 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.204738    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-model-cache\") pod \"simulator-w3-a1617e-kserve-8576757679-9k482\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") " pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.204824 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.204772    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f71923a7-7e8a-48e0-adcd-a061c07eb212-tls-certs\") pod \"simulator-w3-a1617e-kserve-8576757679-9k482\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") " pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.204948 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.204902    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-dshm\") pod \"simulator-w3-a1617e-kserve-8576757679-9k482\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") " pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.204992 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.204947    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-home\") pod \"simulator-w3-a1617e-kserve-8576757679-9k482\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") " pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.205130 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.205101    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv8pt\" (UniqueName: \"kubernetes.io/projected/f71923a7-7e8a-48e0-adcd-a061c07eb212-kube-api-access-sv8pt\") pod \"simulator-w3-a1617e-kserve-8576757679-9k482\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") " pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.270261 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.270230    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-a1617e\"/\"openshift-service-ca.crt\""
Oct 30 18:52:42.270261 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.270231    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-a1617e\"/\"kube-root-ca.crt\""
Oct 30 18:52:42.270261 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.270254    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-a1617e\"/\"default-dockercfg-xbgmq\""
Oct 30 18:52:42.270520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.270439    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-a1617e\"/\"simulator-w3-a1617e-kserve-self-signed-certs\""
Oct 30 18:52:42.272539 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.272517    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"]
Oct 30 18:52:42.306666 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.306638    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-model-cache\") pod \"simulator-w3-a1617e-kserve-8576757679-9k482\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") " pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.306820 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.306678    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f71923a7-7e8a-48e0-adcd-a061c07eb212-tls-certs\") pod \"simulator-w3-a1617e-kserve-8576757679-9k482\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") " pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.306820 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.306736    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-dshm\") pod \"simulator-w3-a1617e-kserve-8576757679-9k482\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") " pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.306820 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.306777    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-home\") pod \"simulator-w3-a1617e-kserve-8576757679-9k482\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") " pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.306982 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.306832    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv8pt\" (UniqueName: \"kubernetes.io/projected/f71923a7-7e8a-48e0-adcd-a061c07eb212-kube-api-access-sv8pt\") pod \"simulator-w3-a1617e-kserve-8576757679-9k482\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") " pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.306982 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.306924    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-tmp-dir\") pod \"simulator-w3-a1617e-kserve-8576757679-9k482\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") " pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.307141 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.307082    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-model-cache\") pod \"simulator-w3-a1617e-kserve-8576757679-9k482\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") " pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.307198 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.307175    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-home\") pod \"simulator-w3-a1617e-kserve-8576757679-9k482\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") " pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.307255 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.307205    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-tmp-dir\") pod \"simulator-w3-a1617e-kserve-8576757679-9k482\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") " pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.309026 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.309003    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-dshm\") pod \"simulator-w3-a1617e-kserve-8576757679-9k482\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") " pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.309553 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.309535    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f71923a7-7e8a-48e0-adcd-a061c07eb212-tls-certs\") pod \"simulator-w3-a1617e-kserve-8576757679-9k482\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") " pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.361995 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.361962    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv8pt\" (UniqueName: \"kubernetes.io/projected/f71923a7-7e8a-48e0-adcd-a061c07eb212-kube-api-access-sv8pt\") pod \"simulator-w3-a1617e-kserve-8576757679-9k482\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") " pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.485563 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.485490    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:42.853076 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:42.853048    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"]
Oct 30 18:52:42.855082 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:52:42.855052    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf71923a7_7e8a_48e0_adcd_a061c07eb212.slice/crio-b2058c2e748453c5d7f932c924363b3a9ddafc59e55e4c70c6ef332707807ef7 WatchSource:0}: Error finding container b2058c2e748453c5d7f932c924363b3a9ddafc59e55e4c70c6ef332707807ef7: Status 404 returned error can't find the container with id b2058c2e748453c5d7f932c924363b3a9ddafc59e55e4c70c6ef332707807ef7
Oct 30 18:52:43.664973 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:43.664937    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" event={"ID":"f71923a7-7e8a-48e0-adcd-a061c07eb212","Type":"ContainerStarted","Data":"12fd8132f4acf9a43d7ca404706a45331655e119bfa08d85dd42aa68e139e905"}
Oct 30 18:52:43.665466 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:43.664984    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" event={"ID":"f71923a7-7e8a-48e0-adcd-a061c07eb212","Type":"ContainerStarted","Data":"b2058c2e748453c5d7f932c924363b3a9ddafc59e55e4c70c6ef332707807ef7"}
Oct 30 18:52:43.774196 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:43.774142    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podStartSLOduration=1.774128228 podStartE2EDuration="1.774128228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:43.772534314 +0000 UTC m=+961.298409411" watchObservedRunningTime="2026-10-30 18:52:43.774128228 +0000 UTC m=+961.300003365"
Oct 30 18:52:44.486028 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:44.485983    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:52.486576 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:52.486542    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:52.499511 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:52.499482    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:52:52.722491 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:52:52.722461    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:53:05.531659 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:53:05.531587    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:53:24.612970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.612940    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"]
Oct 30 18:53:24.616911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.616891    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:24.714068 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.714038    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-3b6ad6\"/\"openshift-service-ca.crt\""
Oct 30 18:53:24.714068 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.714042    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-3b6ad6\"/\"kube-root-ca.crt\""
Oct 30 18:53:24.714068 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.714087    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-3b6ad6\"/\"premium-w5-3b6ad6-kserve-self-signed-certs\""
Oct 30 18:53:24.714320 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.714156    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-3b6ad6\"/\"default-dockercfg-b8plq\""
Oct 30 18:53:24.717349 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.717325    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"]
Oct 30 18:53:24.785015 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.784982    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-model-cache\") pod \"premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") " pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:24.785171 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.785025    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-tmp-dir\") pod \"premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") " pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:24.785171 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.785125    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-dshm\") pod \"premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") " pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:24.785256 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.785170    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmsjm\" (UniqueName: \"kubernetes.io/projected/79574f74-b0c4-4584-8f18-84f1659e020b-kube-api-access-zmsjm\") pod \"premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") " pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:24.785256 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.785205    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-home\") pod \"premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") " pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:24.785374 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.785349    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79574f74-b0c4-4584-8f18-84f1659e020b-tls-certs\") pod \"premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") " pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:24.886810 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.886737    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-dshm\") pod \"premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") " pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:24.886948 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.886809    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmsjm\" (UniqueName: \"kubernetes.io/projected/79574f74-b0c4-4584-8f18-84f1659e020b-kube-api-access-zmsjm\") pod \"premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") " pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:24.886948 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.886843    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-home\") pod \"premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") " pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:24.886948 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.886906    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79574f74-b0c4-4584-8f18-84f1659e020b-tls-certs\") pod \"premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") " pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:24.887091 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.886953    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-model-cache\") pod \"premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") " pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:24.887091 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.886986    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-tmp-dir\") pod \"premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") " pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:24.887325 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.887302    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-home\") pod \"premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") " pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:24.887411 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.887362    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-tmp-dir\") pod \"premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") " pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:24.887475 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.887409    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-model-cache\") pod \"premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") " pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:24.888991 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.888969    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-dshm\") pod \"premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") " pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:24.889445 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.889423    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79574f74-b0c4-4584-8f18-84f1659e020b-tls-certs\") pod \"premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") " pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:24.940579 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:24.940548    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmsjm\" (UniqueName: \"kubernetes.io/projected/79574f74-b0c4-4584-8f18-84f1659e020b-kube-api-access-zmsjm\") pod \"premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") " pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:25.229091 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:25.229007    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:25.609790 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:25.609763    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"]
Oct 30 18:53:25.611878 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:53:25.611849    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79574f74_b0c4_4584_8f18_84f1659e020b.slice/crio-ec476950021d533184d3889c534cffd5f6347c2d77b5fcb4660588bfb44d7a43 WatchSource:0}: Error finding container ec476950021d533184d3889c534cffd5f6347c2d77b5fcb4660588bfb44d7a43: Status 404 returned error can't find the container with id ec476950021d533184d3889c534cffd5f6347c2d77b5fcb4660588bfb44d7a43
Oct 30 18:53:25.887680 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:25.887568    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" event={"ID":"79574f74-b0c4-4584-8f18-84f1659e020b","Type":"ContainerStarted","Data":"cbb186716b76112b2a444ff9e6993a8554edfc0cc22a962fbffe57033842a6dd"}
Oct 30 18:53:25.887680 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:25.887626    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" event={"ID":"79574f74-b0c4-4584-8f18-84f1659e020b","Type":"ContainerStarted","Data":"ec476950021d533184d3889c534cffd5f6347c2d77b5fcb4660588bfb44d7a43"}
Oct 30 18:53:25.997381 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:25.997331    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podStartSLOduration=1.997315934 podStartE2EDuration="1.997315934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:25.996369967 +0000 UTC m=+1003.522245062" watchObservedRunningTime="2026-10-30 18:53:25.997315934 +0000 UTC m=+1003.523191031"
Oct 30 18:53:26.229857 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:26.229780    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:35.229762 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:35.229672    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:35.242895 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:35.242867    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:35.531905 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:53:35.531858    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:53:35.952209 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:35.952141    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:53:39.286733 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.286698    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"]
Oct 30 18:53:39.290839 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.290809    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.338139 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.338102    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-workload-socket\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.338282 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.338215    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-podinfo\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.338282 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.338258    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-workload-certs\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.338389 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.338314    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-credential-socket\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.338452 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.338404    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-token\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.338489 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.338455    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l4t7\" (UniqueName: \"kubernetes.io/projected/9dd18fca-58d5-45d5-8894-1c12386726fa-kube-api-access-9l4t7\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.338637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.338590    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-envoy\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.338690 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.338675    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dd18fca-58d5-45d5-8894-1c12386726fa-istiod-ca-cert\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.338734 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.338708    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-data\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.345791 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.345752    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"]
Oct 30 18:53:39.388818 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.388795    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-3b6ad6-openshift-default-dockercfg-wmgsn\""
Oct 30 18:53:39.439439 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.439410    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l4t7\" (UniqueName: \"kubernetes.io/projected/9dd18fca-58d5-45d5-8894-1c12386726fa-kube-api-access-9l4t7\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.439648 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.439486    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-envoy\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.439648 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.439522    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dd18fca-58d5-45d5-8894-1c12386726fa-istiod-ca-cert\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.439648 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.439556    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-data\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.439648 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.439631    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-workload-socket\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.439895 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.439688    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-podinfo\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.439895 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.439717    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-workload-certs\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.439895 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.439766    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-credential-socket\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.439895 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.439815    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-token\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.440099 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.440022    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-data\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.440156 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.440089    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-workload-socket\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.440156 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.440113    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-workload-certs\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.440300 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.440273    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-credential-socket\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.440425 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.440370    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dd18fca-58d5-45d5-8894-1c12386726fa-istiod-ca-cert\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.442121 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.442101    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-envoy\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.442202 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.442166    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-podinfo\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.495311 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.495282    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-token\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.495439 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.495421    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l4t7\" (UniqueName: \"kubernetes.io/projected/9dd18fca-58d5-45d5-8894-1c12386726fa-kube-api-access-9l4t7\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.605471 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.605385    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:39.951018 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.950931    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"]
Oct 30 18:53:39.956221 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:39.956197    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.008148 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.008112    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-f5705f-openshift-default-dockercfg-hwfjh\""
Oct 30 18:53:40.010109 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.010085    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"]
Oct 30 18:53:40.046735 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.046705    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-credential-socket\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.046887 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.046743    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq7rb\" (UniqueName: \"kubernetes.io/projected/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-kube-api-access-xq7rb\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.046887 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.046761    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-data\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.046887 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.046781    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-token\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.046999 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.046939    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-workload-socket\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.047033 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.046999    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-podinfo\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.047132 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.047116    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-envoy\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.047177 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.047161    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istiod-ca-cert\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.047234 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.047220    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-workload-certs\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.067637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.067578    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"]
Oct 30 18:53:40.104529 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:53:40.104504    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dd18fca_58d5_45d5_8894_1c12386726fa.slice/crio-f94252e68939df7c08583db4ece433cfd689cb1b58f267f93cc376af9381f307 WatchSource:0}: Error finding container f94252e68939df7c08583db4ece433cfd689cb1b58f267f93cc376af9381f307: Status 404 returned error can't find the container with id f94252e68939df7c08583db4ece433cfd689cb1b58f267f93cc376af9381f307
Oct 30 18:53:40.106817 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.106783    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:53:40.106928 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.106862    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:53:40.106928 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.106910    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:53:40.148272 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.148247    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-envoy\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.148367 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.148297    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istiod-ca-cert\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.148417 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.148367    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-workload-certs\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.148488 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.148472    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-credential-socket\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.148526 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.148507    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq7rb\" (UniqueName: \"kubernetes.io/projected/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-kube-api-access-xq7rb\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.148574 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.148537    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-data\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.148654 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.148572    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-token\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.148654 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.148637    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-workload-socket\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.148769 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.148676    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-podinfo\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.148769 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.148754    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-workload-certs\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.148869 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.148846    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-credential-socket\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.149126 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.149103    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istiod-ca-cert\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.149262 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.149240    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-workload-socket\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.149378 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.149356    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-data\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.150881 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.150857    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-envoy\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.151412 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.151391    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-podinfo\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.203198 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.203127    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-token\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.203372 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.203348    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq7rb\" (UniqueName: \"kubernetes.io/projected/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-kube-api-access-xq7rb\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.270426 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.270401    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:40.700962 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.700935    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"]
Oct 30 18:53:40.702630 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:53:40.702584    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4ac89bb_06f0_4b1d_a5f2_98a3d59f37b4.slice/crio-7ffc1a9fe7dc8b907569685b7c84df802d9d692e123213806ec6f2cffca31a61 WatchSource:0}: Error finding container 7ffc1a9fe7dc8b907569685b7c84df802d9d692e123213806ec6f2cffca31a61: Status 404 returned error can't find the container with id 7ffc1a9fe7dc8b907569685b7c84df802d9d692e123213806ec6f2cffca31a61
Oct 30 18:53:40.705149 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.705120    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:53:40.705224 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.705188    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:53:40.705263 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.705221    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:53:40.995686 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.991031    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5" event={"ID":"9dd18fca-58d5-45d5-8894-1c12386726fa","Type":"ContainerStarted","Data":"6b8cd4f8cb980e52b94c6fd6cad639efbf3f3c2c7a1c553c9ee9a8c9f34d2e33"}
Oct 30 18:53:40.995686 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:40.991107    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5" event={"ID":"9dd18fca-58d5-45d5-8894-1c12386726fa","Type":"ContainerStarted","Data":"f94252e68939df7c08583db4ece433cfd689cb1b58f267f93cc376af9381f307"}
Oct 30 18:53:41.002444 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:41.002406    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn" event={"ID":"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4","Type":"ContainerStarted","Data":"14e7557314a12db33fe80d6446599c175210ba979d5778b68d022f52a3aab00b"}
Oct 30 18:53:41.002640 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:41.002469    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn" event={"ID":"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4","Type":"ContainerStarted","Data":"7ffc1a9fe7dc8b907569685b7c84df802d9d692e123213806ec6f2cffca31a61"}
Oct 30 18:53:41.103490 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:41.103434    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5" podStartSLOduration=2.103419992 podStartE2EDuration="2.103419992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:41.101879378 +0000 UTC m=+1018.627754487" watchObservedRunningTime="2026-10-30 18:53:41.103419992 +0000 UTC m=+1018.629295088"
Oct 30 18:53:41.219138 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:41.219086    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn" podStartSLOduration=2.219071163 podStartE2EDuration="2.219071163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:41.217168562 +0000 UTC m=+1018.743043661" watchObservedRunningTime="2026-10-30 18:53:41.219071163 +0000 UTC m=+1018.744946258"
Oct 30 18:53:41.282160 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:41.282041    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:41.284654 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:41.284618    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 18:53:41.284811 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:41.284681    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn" podUID="b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 18:53:41.608686 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:41.607824    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:41.608686 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:41.607999    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 18:53:41.608686 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:41.608045    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5" podUID="9dd18fca-58d5-45d5-8894-1c12386726fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 18:53:42.280626 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:42.280090    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 18:53:42.280626 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:42.280167    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn" podUID="b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 18:53:42.612683 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:42.609709    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 18:53:42.612683 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:42.609772    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5" podUID="9dd18fca-58d5-45d5-8894-1c12386726fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 18:53:43.278625 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:43.278111    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 18:53:43.278625 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:43.278190    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn" podUID="b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 18:53:43.608054 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:43.607961    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 18:53:43.608718 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:43.608687    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5" podUID="9dd18fca-58d5-45d5-8894-1c12386726fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 18:53:44.271993 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:44.271951    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 18:53:44.272278 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:44.272247    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn" podUID="b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 18:53:44.615694 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:44.612955    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 18:53:44.615694 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:44.613016    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5" podUID="9dd18fca-58d5-45d5-8894-1c12386726fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 18:53:45.274014 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:45.273975    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 18:53:45.274322 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:45.274294    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn" podUID="b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 18:53:45.620679 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:45.620113    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 18:53:45.620679 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:45.620178    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5" podUID="9dd18fca-58d5-45d5-8894-1c12386726fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 18:53:46.294643 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:46.275739    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 18:53:46.295433 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:46.294986    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn" podUID="b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 18:53:46.606552 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:46.606469    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 30 18:53:46.606785 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:46.606757    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5" podUID="9dd18fca-58d5-45d5-8894-1c12386726fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 30 18:53:47.271095 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:47.271056    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 30 18:53:47.271615 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:47.271139    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn" podUID="b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 30 18:53:47.610241 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:47.610160    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:47.610452 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:47.610429    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:47.611288 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:47.611271    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 18:53:48.275292 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:48.275261    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:48.275787 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:48.275637    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:53:48.276456 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:53:48.276435    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 18:54:05.531296 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:54:05.531217    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:54:23.983489 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:23.983445    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"]
Oct 30 18:54:23.987819 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:23.987797    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.039734 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.039702    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-649d16-openshift-default-dockercfg-xgvdr\""
Oct 30 18:54:24.076163 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.076133    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-envoy\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.076331 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.076184    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-workload-socket\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.076331 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.076271    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-credential-socket\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.076331 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.076310    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istiod-ca-cert\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.076525 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.076336    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-token\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.076525 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.076443    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-data\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.076525 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.076469    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-podinfo\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.076722 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.076526    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz68p\" (UniqueName: \"kubernetes.io/projected/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-kube-api-access-zz68p\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.076722 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.076621    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-workload-certs\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.099295 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.099243    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"]
Oct 30 18:54:24.181329 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.181283    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz68p\" (UniqueName: \"kubernetes.io/projected/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-kube-api-access-zz68p\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.181549 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.181365    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-workload-certs\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.181549 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.181432    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-envoy\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.181549 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.181469    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-workload-socket\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.181549 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.181511    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-credential-socket\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.181909 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.181551    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istiod-ca-cert\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.181909 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.181590    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-token\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.181909 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.181696    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-data\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.181909 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.181742    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-podinfo\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.182800 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.182770    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-workload-certs\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.183689 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.183662    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-credential-socket\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.184188 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.184160    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-data\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.184921 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.184895    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istiod-ca-cert\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.185214 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.185189    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-workload-socket\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.186056 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.186032    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-podinfo\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.220659 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.216721    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-envoy\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.239755 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.239693    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-token\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.239755 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.239736    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz68p\" (UniqueName: \"kubernetes.io/projected/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-kube-api-access-zz68p\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.302745 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.300592    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"]
Oct 30 18:54:24.304476 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.304451    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:24.314530 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.314511    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.345696 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.345589    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"]
Oct 30 18:54:24.410708 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.410670    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-credential-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.410833 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.410760    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-workload-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.410833 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.410805    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-workload-certs\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.410954 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.410873    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng6k8\" (UniqueName: \"kubernetes.io/projected/34e14ce3-0d46-4301-b047-43b7ddc23088-kube-api-access-ng6k8\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.410954 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.410921    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-podinfo\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.411065 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.410968    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-data\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.411065 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.411008    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-token\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.411171 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.411103    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-envoy\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.411245 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.411222    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34e14ce3-0d46-4301-b047-43b7ddc23088-istiod-ca-cert\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.518751 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.518713    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-workload-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.518934 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.518765    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-workload-certs\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.518934 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.518816    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng6k8\" (UniqueName: \"kubernetes.io/projected/34e14ce3-0d46-4301-b047-43b7ddc23088-kube-api-access-ng6k8\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.518934 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.518863    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-podinfo\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.518934 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.518899    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-data\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.519175 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.518938    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-token\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.519175 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.519015    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-envoy\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.519175 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.519123    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34e14ce3-0d46-4301-b047-43b7ddc23088-istiod-ca-cert\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.519347 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.519188    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-credential-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.519347 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.519315    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-workload-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.521283 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.521251    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-credential-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.522528 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.521711    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-data\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.522528 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.521954    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-workload-certs\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.526788 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.524402    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-envoy\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.526788 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.524983    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34e14ce3-0d46-4301-b047-43b7ddc23088-istiod-ca-cert\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.527507 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.527461    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-podinfo\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.576252 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.576207    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-token\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.576841 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.576813    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng6k8\" (UniqueName: \"kubernetes.io/projected/34e14ce3-0d46-4301-b047-43b7ddc23088-kube-api-access-ng6k8\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.650308 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.650273    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:24.763413 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.763309    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"]
Oct 30 18:54:24.766093 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:54:24.766067    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34f008d5_e8ec_48d2_9ffb_1d00ceb1f9fd.slice/crio-72cda0569f9ea4db4b274c7027a46e9f9d08fc0949b0b9203ebb5baf0f2d0563 WatchSource:0}: Error finding container 72cda0569f9ea4db4b274c7027a46e9f9d08fc0949b0b9203ebb5baf0f2d0563: Status 404 returned error can't find the container with id 72cda0569f9ea4db4b274c7027a46e9f9d08fc0949b0b9203ebb5baf0f2d0563
Oct 30 18:54:24.769258 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.769182    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:54:24.769365 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.769255    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:54:24.769365 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:24.769290    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:54:25.021180 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:25.021151    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"]
Oct 30 18:54:25.022936 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:54:25.022911    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34e14ce3_0d46_4301_b047_43b7ddc23088.slice/crio-d839c4119549cd76c8df31e4c0f48224cc427fd29fd00f404ee142af45877b25 WatchSource:0}: Error finding container d839c4119549cd76c8df31e4c0f48224cc427fd29fd00f404ee142af45877b25: Status 404 returned error can't find the container with id d839c4119549cd76c8df31e4c0f48224cc427fd29fd00f404ee142af45877b25
Oct 30 18:54:25.025463 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:25.025431    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:54:25.025577 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:25.025507    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:54:25.025577 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:25.025552    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:54:25.299544 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:25.299448    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb" event={"ID":"34e14ce3-0d46-4301-b047-43b7ddc23088","Type":"ContainerStarted","Data":"c7573282c64be82db694a1731b657a872eab568a11076d920469fd5a7cac5aae"}
Oct 30 18:54:25.299544 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:25.299514    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb" event={"ID":"34e14ce3-0d46-4301-b047-43b7ddc23088","Type":"ContainerStarted","Data":"d839c4119549cd76c8df31e4c0f48224cc427fd29fd00f404ee142af45877b25"}
Oct 30 18:54:25.301242 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:25.301217    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg" event={"ID":"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd","Type":"ContainerStarted","Data":"9bd2c6d9961d184842235dfcc10cea15730d9af3adb6ebfcb330e0ceafcba44f"}
Oct 30 18:54:25.301349 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:25.301256    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg" event={"ID":"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd","Type":"ContainerStarted","Data":"72cda0569f9ea4db4b274c7027a46e9f9d08fc0949b0b9203ebb5baf0f2d0563"}
Oct 30 18:54:25.305642 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:25.305591    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:25.307714 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:25.307690    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 18:54:25.307810 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:25.307734    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg" podUID="34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 18:54:25.410584 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:25.410529    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb" podStartSLOduration=1.410512747 podStartE2EDuration="1.410512747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:25.409257306 +0000 UTC m=+1062.935132402" watchObservedRunningTime="2026-10-30 18:54:25.410512747 +0000 UTC m=+1062.936387843"
Oct 30 18:54:25.532573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:25.532516    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg" podStartSLOduration=2.532499823 podStartE2EDuration="2.532499823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:25.532062662 +0000 UTC m=+1063.057937741" watchObservedRunningTime="2026-10-30 18:54:25.532499823 +0000 UTC m=+1063.058374919"
Oct 30 18:54:25.650992 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:25.650918    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:26.315186 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:26.315048    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 18:54:26.315186 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:26.315104    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg" podUID="34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 18:54:26.652093 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:26.652012    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 18:54:26.652093 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:26.652062    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb" podUID="34e14ce3-0d46-4301-b047-43b7ddc23088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 18:54:27.306017 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:27.305981    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 18:54:27.306308 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:27.306281    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg" podUID="34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 18:54:27.651240 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:27.651167    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 18:54:27.651240 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:27.651221    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb" podUID="34e14ce3-0d46-4301-b047-43b7ddc23088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 18:54:28.308236 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:28.308200    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 18:54:28.308503 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:28.308475    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg" podUID="34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 18:54:28.651434 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:28.651356    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 18:54:28.651434 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:28.651409    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb" podUID="34e14ce3-0d46-4301-b047-43b7ddc23088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 18:54:29.307627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:29.306915    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 18:54:29.307627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:29.306977    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg" podUID="34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 18:54:29.651027 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:29.650944    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 18:54:29.651027 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:29.650993    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb" podUID="34e14ce3-0d46-4301-b047-43b7ddc23088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 18:54:30.306561 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:30.306044    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 18:54:30.306561 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:30.306128    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg" podUID="34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 18:54:30.651259 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:30.651170    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Oct 30 18:54:30.651259 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:30.651229    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb" podUID="34e14ce3-0d46-4301-b047-43b7ddc23088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Oct 30 18:54:31.305258 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:31.305218    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 30 18:54:31.305451 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:31.305274    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg" podUID="34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 30 18:54:31.853216 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:31.853182    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:31.853772 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:31.853750    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:31.854675 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:31.854653    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 18:54:32.309653 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:32.309621    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:32.310115 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:32.310093    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:32.310761 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:32.310742    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 18:54:35.532326 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:54:35.532286    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:54:45.768372 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:45.768333    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"]
Oct 30 18:54:45.769194 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:45.768916    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6" podUID="7bfbe33e-bf31-44f9-95ec-812a39171e94" containerName="istio-proxy" containerID="cri-o://d3c299b1d24f4d071186dfb41bd254263a793c9f1f538ddfd2902497bb618679" gracePeriod=30
Oct 30 18:54:51.035341 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.035279    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:54:51.178697 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.178658    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-token\" (UniqueName: \"kubernetes.io/projected/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-token\") pod \"7bfbe33e-bf31-44f9-95ec-812a39171e94\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") "
Oct 30 18:54:51.178892 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.178742    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-data\") pod \"7bfbe33e-bf31-44f9-95ec-812a39171e94\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") "
Oct 30 18:54:51.178892 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.178807    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-envoy\") pod \"7bfbe33e-bf31-44f9-95ec-812a39171e94\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") "
Oct 30 18:54:51.178892 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.178849    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bfbe33e-bf31-44f9-95ec-812a39171e94-kube-api-access-7d2dn\" (UniqueName: \"kubernetes.io/projected/7bfbe33e-bf31-44f9-95ec-812a39171e94-kube-api-access-7d2dn\") pod \"7bfbe33e-bf31-44f9-95ec-812a39171e94\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") "
Oct 30 18:54:51.179068 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.178935    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7bfbe33e-bf31-44f9-95ec-812a39171e94-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bfbe33e-bf31-44f9-95ec-812a39171e94-istiod-ca-cert\") pod \"7bfbe33e-bf31-44f9-95ec-812a39171e94\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") "
Oct 30 18:54:51.179068 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.178988    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-workload-socket\") pod \"7bfbe33e-bf31-44f9-95ec-812a39171e94\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") "
Oct 30 18:54:51.179068 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.179009    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-data" pod "7bfbe33e-bf31-44f9-95ec-812a39171e94" (UID: "7bfbe33e-bf31-44f9-95ec-812a39171e94"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:54:51.179068 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.179036    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-podinfo\") pod \"7bfbe33e-bf31-44f9-95ec-812a39171e94\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") "
Oct 30 18:54:51.179275 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.179097    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-credential-socket\") pod \"7bfbe33e-bf31-44f9-95ec-812a39171e94\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") "
Oct 30 18:54:51.179275 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.179132    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-workload-certs\") pod \"7bfbe33e-bf31-44f9-95ec-812a39171e94\" (UID: \"7bfbe33e-bf31-44f9-95ec-812a39171e94\") "
Oct 30 18:54:51.179275 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.179236    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-workload-socket" pod "7bfbe33e-bf31-44f9-95ec-812a39171e94" (UID: "7bfbe33e-bf31-44f9-95ec-812a39171e94"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:54:51.179434 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.179320    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bfbe33e-bf31-44f9-95ec-812a39171e94-istiod-ca-cert" pod "7bfbe33e-bf31-44f9-95ec-812a39171e94" (UID: "7bfbe33e-bf31-44f9-95ec-812a39171e94"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:54:51.179583 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.179552    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-workload-certs" pod "7bfbe33e-bf31-44f9-95ec-812a39171e94" (UID: "7bfbe33e-bf31-44f9-95ec-812a39171e94"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:54:51.179771 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.179698    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-credential-socket" pod "7bfbe33e-bf31-44f9-95ec-812a39171e94" (UID: "7bfbe33e-bf31-44f9-95ec-812a39171e94"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:54:51.179836 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.179813    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:51.179889 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.179835    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bfbe33e-bf31-44f9-95ec-812a39171e94-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:51.179889 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.179850    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:51.179889 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.179865    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:51.179889 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.179878    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:51.181538 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.181514    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-podinfo" pod "7bfbe33e-bf31-44f9-95ec-812a39171e94" (UID: "7bfbe33e-bf31-44f9-95ec-812a39171e94"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:54:51.181538 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.181530    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-token" pod "7bfbe33e-bf31-44f9-95ec-812a39171e94" (UID: "7bfbe33e-bf31-44f9-95ec-812a39171e94"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:51.181899 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.181879    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bfbe33e-bf31-44f9-95ec-812a39171e94-kube-api-access-7d2dn" pod "7bfbe33e-bf31-44f9-95ec-812a39171e94" (UID: "7bfbe33e-bf31-44f9-95ec-812a39171e94"). InnerVolumeSpecName "kube-api-access-7d2dn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:51.181983 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.181907    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-envoy" pod "7bfbe33e-bf31-44f9-95ec-812a39171e94" (UID: "7bfbe33e-bf31-44f9-95ec-812a39171e94"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:54:51.281254 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.281225    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:51.281254 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.281252    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:51.281254 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.281262    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bfbe33e-bf31-44f9-95ec-812a39171e94-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:51.281478 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.281272    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7d2dn\" (UniqueName: \"kubernetes.io/projected/7bfbe33e-bf31-44f9-95ec-812a39171e94-kube-api-access-7d2dn\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:51.457412 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.457331    4544 generic.go:356] "Generic (PLEG): container finished" podID="7bfbe33e-bf31-44f9-95ec-812a39171e94" containerID="d3c299b1d24f4d071186dfb41bd254263a793c9f1f538ddfd2902497bb618679" exitCode=0
Oct 30 18:54:51.457574 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.457420    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6" event={"ID":"7bfbe33e-bf31-44f9-95ec-812a39171e94","Type":"ContainerDied","Data":"d3c299b1d24f4d071186dfb41bd254263a793c9f1f538ddfd2902497bb618679"}
Oct 30 18:54:51.457574 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.457494    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6" event={"ID":"7bfbe33e-bf31-44f9-95ec-812a39171e94","Type":"ContainerDied","Data":"3f1d5ee36a3f381838296c59dae84d20436dc69a5dff786997a849cccffb03db"}
Oct 30 18:54:51.457574 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.457526    4544 scope.go:122] "RemoveContainer" containerID="d3c299b1d24f4d071186dfb41bd254263a793c9f1f538ddfd2902497bb618679"
Oct 30 18:54:51.457798 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.457776    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"
Oct 30 18:54:51.495817 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.495788    4544 scope.go:122] "RemoveContainer" containerID="d3c299b1d24f4d071186dfb41bd254263a793c9f1f538ddfd2902497bb618679"
Oct 30 18:54:51.496251 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:54:51.496214    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3c299b1d24f4d071186dfb41bd254263a793c9f1f538ddfd2902497bb618679\": container with ID starting with d3c299b1d24f4d071186dfb41bd254263a793c9f1f538ddfd2902497bb618679 not found: ID does not exist" containerID="d3c299b1d24f4d071186dfb41bd254263a793c9f1f538ddfd2902497bb618679"
Oct 30 18:54:51.496372 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.496266    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3c299b1d24f4d071186dfb41bd254263a793c9f1f538ddfd2902497bb618679"} err="failed to get container status \"d3c299b1d24f4d071186dfb41bd254263a793c9f1f538ddfd2902497bb618679\": rpc error: code = NotFound desc = could not find container \"d3c299b1d24f4d071186dfb41bd254263a793c9f1f538ddfd2902497bb618679\": container with ID starting with d3c299b1d24f4d071186dfb41bd254263a793c9f1f538ddfd2902497bb618679 not found: ID does not exist"
Oct 30 18:54:51.614141 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.614068    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"]
Oct 30 18:54:51.660251 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:51.660218    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-lwkt6"]
Oct 30 18:54:52.603215 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:52.603179    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7bfbe33e-bf31-44f9-95ec-812a39171e94" path="/var/lib/kubelet/pods/7bfbe33e-bf31-44f9-95ec-812a39171e94/volumes"
Oct 30 18:54:55.265700 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.265123    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"]
Oct 30 18:54:55.266330 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.265957    4544 cpu_manager.go:414] "containerMap: removing container" podUID="7bfbe33e-bf31-44f9-95ec-812a39171e94" containerName="istio-proxy"
Oct 30 18:54:55.266330 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.265979    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7bfbe33e-bf31-44f9-95ec-812a39171e94" containerName="istio-proxy"
Oct 30 18:54:55.266330 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.266091    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="7bfbe33e-bf31-44f9-95ec-812a39171e94" containerName="istio-proxy"
Oct 30 18:54:55.286720 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.284769    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.352059 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.351118    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"]
Oct 30 18:54:55.386624 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.386534    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-a1617e-openshift-default-dockercfg-fbsx5\""
Oct 30 18:54:55.452827 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.452797    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-workload-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.452977 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.452882    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-credential-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.452977 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.452920    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-workload-certs\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.453055 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.452991    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-envoy\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.453055 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.453021    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pkbs\" (UniqueName: \"kubernetes.io/projected/64da4d53-629c-4015-aaf9-2e3765f9dc5c-kube-api-access-7pkbs\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.453129 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.453062    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-data\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.453183 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.453165    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istiod-ca-cert\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.453227 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.453192    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-token\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.453227 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.453213    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-podinfo\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.554750 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.554672    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-data\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.554914 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.554756    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istiod-ca-cert\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.554914 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.554782    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-token\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.554914 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.554813    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-podinfo\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.555082 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.554937    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-workload-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.555082 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.554981    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-credential-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.555082 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.555016    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-workload-certs\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.555082 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.555061    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-envoy\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.555274 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.555096    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pkbs\" (UniqueName: \"kubernetes.io/projected/64da4d53-629c-4015-aaf9-2e3765f9dc5c-kube-api-access-7pkbs\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.555274 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.555106    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-data\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.555363 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.555322    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-workload-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.555418 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.555367    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-credential-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.555492 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.555470    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istiod-ca-cert\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.555637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.555599    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-workload-certs\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.570626 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.557628    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-podinfo\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.589890 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.589828    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-envoy\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.630492 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.630458    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pkbs\" (UniqueName: \"kubernetes.io/projected/64da4d53-629c-4015-aaf9-2e3765f9dc5c-kube-api-access-7pkbs\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.631182 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.631072    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-token\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:55.907566 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:55.907479    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:56.340082 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:56.340054    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"]
Oct 30 18:54:56.342196 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:54:56.342165    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64da4d53_629c_4015_aaf9_2e3765f9dc5c.slice/crio-a655b18602c45c3a45e6077b5c5d0cec49277aec8ddfd97a910f749226f03e30 WatchSource:0}: Error finding container a655b18602c45c3a45e6077b5c5d0cec49277aec8ddfd97a910f749226f03e30: Status 404 returned error can't find the container with id a655b18602c45c3a45e6077b5c5d0cec49277aec8ddfd97a910f749226f03e30
Oct 30 18:54:56.344798 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:56.344755    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:54:56.344889 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:56.344841    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:54:56.344889 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:56.344880    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:54:56.496970 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:56.496927    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2" event={"ID":"64da4d53-629c-4015-aaf9-2e3765f9dc5c","Type":"ContainerStarted","Data":"86146affac30927d6614b5e1e11760e83e883478fe7fe2b8b404d8386332f0a7"}
Oct 30 18:54:56.497127 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:56.496986    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2" event={"ID":"64da4d53-629c-4015-aaf9-2e3765f9dc5c","Type":"ContainerStarted","Data":"a655b18602c45c3a45e6077b5c5d0cec49277aec8ddfd97a910f749226f03e30"}
Oct 30 18:54:56.612049 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:56.611944    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2" podStartSLOduration=1.611924577 podStartE2EDuration="1.611924577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:56.60729166 +0000 UTC m=+1094.133166756" watchObservedRunningTime="2026-10-30 18:54:56.611924577 +0000 UTC m=+1094.137799674"
Oct 30 18:54:56.907951 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:56.907864    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:54:57.914628 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:57.913995    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 18:54:57.914628 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:57.914050    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2" podUID="64da4d53-629c-4015-aaf9-2e3765f9dc5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 18:54:58.908119 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:58.908069    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 18:54:58.908296 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:58.908150    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2" podUID="64da4d53-629c-4015-aaf9-2e3765f9dc5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 18:54:59.908506 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:59.908469    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 18:54:59.908932 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:54:59.908535    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2" podUID="64da4d53-629c-4015-aaf9-2e3765f9dc5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 18:55:00.908792 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:00.908761    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 18:55:00.909212 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:00.908813    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2" podUID="64da4d53-629c-4015-aaf9-2e3765f9dc5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 18:55:01.908856 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:01.908823    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 18:55:01.909252 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:01.908877    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2" podUID="64da4d53-629c-4015-aaf9-2e3765f9dc5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 18:55:03.047952 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:03.047917    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:55:03.049110 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:03.049075    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:55:03.050680 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:03.050654    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 18:55:05.533154 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:55:05.533105    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:55:35.530157 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:55:35.530125    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:55:40.409732 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.409698    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"]
Oct 30 18:55:40.414118 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.414095    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.480027 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.479992    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"]
Oct 30 18:55:40.554829 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.554790    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-token\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.555001 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.554852    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssk5s\" (UniqueName: \"kubernetes.io/projected/48229a00-0acf-4c2b-b14e-d0151692b2d1-kube-api-access-ssk5s\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.555001 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.554942    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-workload-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.555001 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.554975    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48229a00-0acf-4c2b-b14e-d0151692b2d1-istiod-ca-cert\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.555001 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.554997    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-data\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.555171 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.555012    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-podinfo\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.555171 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.555121    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-credential-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.555171 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.555143    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-envoy\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.555297 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.555255    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-workload-certs\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.656269 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.656238    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-workload-certs\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.656451 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.656293    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-token\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.656451 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.656328    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssk5s\" (UniqueName: \"kubernetes.io/projected/48229a00-0acf-4c2b-b14e-d0151692b2d1-kube-api-access-ssk5s\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.656451 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.656357    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-workload-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.656451 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.656374    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48229a00-0acf-4c2b-b14e-d0151692b2d1-istiod-ca-cert\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.656451 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.656394    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-data\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.656451 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.656418    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-podinfo\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.656811 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.656485    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-credential-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.656811 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.656522    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-envoy\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.656811 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.656694    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-workload-certs\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.656998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.656973    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-data\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.657063 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.657036    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-credential-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.657119 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.657106    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-workload-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.657306 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.657280    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48229a00-0acf-4c2b-b14e-d0151692b2d1-istiod-ca-cert\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.659029 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.659004    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-envoy\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.659245 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.659228    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-podinfo\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.710100 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.710041    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-token\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.710213 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.710100    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssk5s\" (UniqueName: \"kubernetes.io/projected/48229a00-0acf-4c2b-b14e-d0151692b2d1-kube-api-access-ssk5s\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:40.728091 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:40.728066    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:41.149220 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:41.149185    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"]
Oct 30 18:55:41.187058 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:55:41.187027    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48229a00_0acf_4c2b_b14e_d0151692b2d1.slice/crio-18ad5fa11d98d419cf3a59e8dade358ac41a864ccf7d99357b9c1ea3a4d049d5 WatchSource:0}: Error finding container 18ad5fa11d98d419cf3a59e8dade358ac41a864ccf7d99357b9c1ea3a4d049d5: Status 404 returned error can't find the container with id 18ad5fa11d98d419cf3a59e8dade358ac41a864ccf7d99357b9c1ea3a4d049d5
Oct 30 18:55:41.189819 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:41.189783    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:55:41.189943 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:41.189865    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:55:41.189943 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:41.189911    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:55:41.803056 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:41.803016    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7" event={"ID":"48229a00-0acf-4c2b-b14e-d0151692b2d1","Type":"ContainerStarted","Data":"2236c1e02356ff9b0ede8ace5d7bac7d22377206c0f33ead222556ae47d55fdb"}
Oct 30 18:55:41.803570 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:41.803116    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7" event={"ID":"48229a00-0acf-4c2b-b14e-d0151692b2d1","Type":"ContainerStarted","Data":"18ad5fa11d98d419cf3a59e8dade358ac41a864ccf7d99357b9c1ea3a4d049d5"}
Oct 30 18:55:41.999445 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:41.999387    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7" podStartSLOduration=1.9993733200000001 podStartE2EDuration="1.99937332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:41.998561627 +0000 UTC m=+1139.524436726" watchObservedRunningTime="2026-10-30 18:55:41.99937332 +0000 UTC m=+1139.525248416"
Oct 30 18:55:42.729097 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:42.729063    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:42.730753 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:42.730727    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 18:55:42.730899 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:42.730783    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7" podUID="48229a00-0acf-4c2b-b14e-d0151692b2d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 18:55:43.729003 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:43.728970    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 18:55:43.729392 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:43.729033    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7" podUID="48229a00-0acf-4c2b-b14e-d0151692b2d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 18:55:44.728561 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:44.728525    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 18:55:44.728776 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:44.728578    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7" podUID="48229a00-0acf-4c2b-b14e-d0151692b2d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 18:55:45.728491 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:45.728459    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 18:55:45.728911 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:45.728515    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7" podUID="48229a00-0acf-4c2b-b14e-d0151692b2d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 18:55:46.728579 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:46.728540    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 18:55:46.729099 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:46.728642    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7" podUID="48229a00-0acf-4c2b-b14e-d0151692b2d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 18:55:47.728564 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:47.728529    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 30 18:55:47.728810 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:47.728587    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7" podUID="48229a00-0acf-4c2b-b14e-d0151692b2d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 30 18:55:48.733291 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:48.733259    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:48.733715 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:48.733522    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:48.734276 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:48.734256    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 18:55:55.935200 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:55.935168    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"]
Oct 30 18:55:55.940567 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:55.940544    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:55.994121 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:55.994075    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"]
Oct 30 18:55:56.036505 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.036471    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-75e91b\"/\"kube-root-ca.crt\""
Oct 30 18:55:56.036505 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.036472    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-75e91b\"/\"openshift-service-ca.crt\""
Oct 30 18:55:56.036754 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.036736    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-75e91b\"/\"default-dockercfg-qrgx8\""
Oct 30 18:55:56.036811 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.036796    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-75e91b\"/\"distinct-w4-75e91b-kserve-self-signed-certs\""
Oct 30 18:55:56.041224 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.041202    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-home\") pod \"distinct-w4-75e91b-kserve-595cf9cb76-l6lcw\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.041274 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.041262    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngnnj\" (UniqueName: \"kubernetes.io/projected/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-kube-api-access-ngnnj\") pod \"distinct-w4-75e91b-kserve-595cf9cb76-l6lcw\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.041375 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.041362    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-dshm\") pod \"distinct-w4-75e91b-kserve-595cf9cb76-l6lcw\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.041419 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.041392    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-tmp-dir\") pod \"distinct-w4-75e91b-kserve-595cf9cb76-l6lcw\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.041458 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.041428    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-model-cache\") pod \"distinct-w4-75e91b-kserve-595cf9cb76-l6lcw\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.041645 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.041592    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-tls-certs\") pod \"distinct-w4-75e91b-kserve-595cf9cb76-l6lcw\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.142523 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.142484    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-model-cache\") pod \"distinct-w4-75e91b-kserve-595cf9cb76-l6lcw\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.142765 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.142570    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-tls-certs\") pod \"distinct-w4-75e91b-kserve-595cf9cb76-l6lcw\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.142765 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.142632    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-home\") pod \"distinct-w4-75e91b-kserve-595cf9cb76-l6lcw\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.142765 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.142691    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngnnj\" (UniqueName: \"kubernetes.io/projected/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-kube-api-access-ngnnj\") pod \"distinct-w4-75e91b-kserve-595cf9cb76-l6lcw\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.142765 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.142760    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-dshm\") pod \"distinct-w4-75e91b-kserve-595cf9cb76-l6lcw\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.142989 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.142798    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-tmp-dir\") pod \"distinct-w4-75e91b-kserve-595cf9cb76-l6lcw\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.143048 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.142979    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-model-cache\") pod \"distinct-w4-75e91b-kserve-595cf9cb76-l6lcw\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.143153 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.143132    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-tmp-dir\") pod \"distinct-w4-75e91b-kserve-595cf9cb76-l6lcw\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.143293 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.143272    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-home\") pod \"distinct-w4-75e91b-kserve-595cf9cb76-l6lcw\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.145134 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.145104    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-dshm\") pod \"distinct-w4-75e91b-kserve-595cf9cb76-l6lcw\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.145315 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.145294    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-tls-certs\") pod \"distinct-w4-75e91b-kserve-595cf9cb76-l6lcw\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.198123 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.198059    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngnnj\" (UniqueName: \"kubernetes.io/projected/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-kube-api-access-ngnnj\") pod \"distinct-w4-75e91b-kserve-595cf9cb76-l6lcw\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.254091 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.254064    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:55:56.627558 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.627521    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"]
Oct 30 18:55:56.628644 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:55:56.628618    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbfb6e108_03a3_4d0f_94d4_2a29eee0d582.slice/crio-d2a623b25f0269edc2c9939819667598511c016e9d9f092a0de3fd7f91e5c37c WatchSource:0}: Error finding container d2a623b25f0269edc2c9939819667598511c016e9d9f092a0de3fd7f91e5c37c: Status 404 returned error can't find the container with id d2a623b25f0269edc2c9939819667598511c016e9d9f092a0de3fd7f91e5c37c
Oct 30 18:55:56.895712 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.895637    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" event={"ID":"bfb6e108-03a3-4d0f-94d4-2a29eee0d582","Type":"ContainerStarted","Data":"838fdf2a3794916d3a6aaa58be7f6f9de4d3ce49af14069a5843ebaecb1d37d8"}
Oct 30 18:55:56.895712 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:56.895678    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" event={"ID":"bfb6e108-03a3-4d0f-94d4-2a29eee0d582","Type":"ContainerStarted","Data":"d2a623b25f0269edc2c9939819667598511c016e9d9f092a0de3fd7f91e5c37c"}
Oct 30 18:55:57.028665 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:57.025571    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podStartSLOduration=2.025552121 podStartE2EDuration="2.025552121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:57.025195782 +0000 UTC m=+1154.551070878" watchObservedRunningTime="2026-10-30 18:55:57.025552121 +0000 UTC m=+1154.551427218"
Oct 30 18:55:57.254576 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:55:57.254491    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:56:05.531650 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:56:05.531593    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:56:06.254456 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:06.254428    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:56:06.267229 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:06.267204    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:56:06.965646 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:06.965615    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:56:16.526261 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:16.526230    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"]
Oct 30 18:56:16.526818 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:16.526442    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw" podUID="2aaac2d4-38ca-46d5-95ea-52356e01647f" containerName="istio-proxy" containerID="cri-o://48634614b12a7446f65c2d0af746b484963bf34fd8daba368a44839525984fe3" gracePeriod=30
Oct 30 18:56:21.784233 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.784206    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:56:21.860068 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.859977    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2aaac2d4-38ca-46d5-95ea-52356e01647f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2aaac2d4-38ca-46d5-95ea-52356e01647f-istiod-ca-cert\") pod \"2aaac2d4-38ca-46d5-95ea-52356e01647f\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") "
Oct 30 18:56:21.860068 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.860047    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-envoy\") pod \"2aaac2d4-38ca-46d5-95ea-52356e01647f\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") "
Oct 30 18:56:21.860289 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.860110    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-token\" (UniqueName: \"kubernetes.io/projected/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-token\") pod \"2aaac2d4-38ca-46d5-95ea-52356e01647f\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") "
Oct 30 18:56:21.860289 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.860143    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-workload-socket\") pod \"2aaac2d4-38ca-46d5-95ea-52356e01647f\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") "
Oct 30 18:56:21.860289 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.860241    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-podinfo\") pod \"2aaac2d4-38ca-46d5-95ea-52356e01647f\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") "
Oct 30 18:56:21.860442 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.860288    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-data\") pod \"2aaac2d4-38ca-46d5-95ea-52356e01647f\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") "
Oct 30 18:56:21.860442 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.860316    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-credential-socket\") pod \"2aaac2d4-38ca-46d5-95ea-52356e01647f\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") "
Oct 30 18:56:21.860442 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.860343    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2aaac2d4-38ca-46d5-95ea-52356e01647f-istiod-ca-cert" pod "2aaac2d4-38ca-46d5-95ea-52356e01647f" (UID: "2aaac2d4-38ca-46d5-95ea-52356e01647f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:21.860442 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.860359    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2aaac2d4-38ca-46d5-95ea-52356e01647f-kube-api-access-kt8pv\" (UniqueName: \"kubernetes.io/projected/2aaac2d4-38ca-46d5-95ea-52356e01647f-kube-api-access-kt8pv\") pod \"2aaac2d4-38ca-46d5-95ea-52356e01647f\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") "
Oct 30 18:56:21.860442 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.860421    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-workload-certs\") pod \"2aaac2d4-38ca-46d5-95ea-52356e01647f\" (UID: \"2aaac2d4-38ca-46d5-95ea-52356e01647f\") "
Oct 30 18:56:21.860706 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.860628    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-workload-socket" pod "2aaac2d4-38ca-46d5-95ea-52356e01647f" (UID: "2aaac2d4-38ca-46d5-95ea-52356e01647f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:56:21.860765 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.860703    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-credential-socket" pod "2aaac2d4-38ca-46d5-95ea-52356e01647f" (UID: "2aaac2d4-38ca-46d5-95ea-52356e01647f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:56:21.860978 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.860873    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-workload-certs" pod "2aaac2d4-38ca-46d5-95ea-52356e01647f" (UID: "2aaac2d4-38ca-46d5-95ea-52356e01647f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:56:21.861093 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.861018    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-data" pod "2aaac2d4-38ca-46d5-95ea-52356e01647f" (UID: "2aaac2d4-38ca-46d5-95ea-52356e01647f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:56:21.861155 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.861143    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:21.861211 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.861161    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:21.861211 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.861179    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:21.861211 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.861193    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2aaac2d4-38ca-46d5-95ea-52356e01647f-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:21.861211 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.861208    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:21.863447 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.863382    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-token" pod "2aaac2d4-38ca-46d5-95ea-52356e01647f" (UID: "2aaac2d4-38ca-46d5-95ea-52356e01647f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:21.863447 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.863388    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-envoy" pod "2aaac2d4-38ca-46d5-95ea-52356e01647f" (UID: "2aaac2d4-38ca-46d5-95ea-52356e01647f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:56:21.863729 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.863704    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2aaac2d4-38ca-46d5-95ea-52356e01647f-kube-api-access-kt8pv" pod "2aaac2d4-38ca-46d5-95ea-52356e01647f" (UID: "2aaac2d4-38ca-46d5-95ea-52356e01647f"). InnerVolumeSpecName "kube-api-access-kt8pv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:21.864069 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.864005    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-podinfo" pod "2aaac2d4-38ca-46d5-95ea-52356e01647f" (UID: "2aaac2d4-38ca-46d5-95ea-52356e01647f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:56:21.962158 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.962124    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:21.962158 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.962157    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:21.962385 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.962172    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kt8pv\" (UniqueName: \"kubernetes.io/projected/2aaac2d4-38ca-46d5-95ea-52356e01647f-kube-api-access-kt8pv\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:21.962385 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:21.962186    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2aaac2d4-38ca-46d5-95ea-52356e01647f-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:22.047373 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.047335    4544 generic.go:356] "Generic (PLEG): container finished" podID="2aaac2d4-38ca-46d5-95ea-52356e01647f" containerID="48634614b12a7446f65c2d0af746b484963bf34fd8daba368a44839525984fe3" exitCode=0
Oct 30 18:56:22.047536 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.047387    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"
Oct 30 18:56:22.047536 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.047421    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw" event={"ID":"2aaac2d4-38ca-46d5-95ea-52356e01647f","Type":"ContainerDied","Data":"48634614b12a7446f65c2d0af746b484963bf34fd8daba368a44839525984fe3"}
Oct 30 18:56:22.047536 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.047478    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw" event={"ID":"2aaac2d4-38ca-46d5-95ea-52356e01647f","Type":"ContainerDied","Data":"811d0acb2854f1d285c4e87de544af0b92db978cde7ca19019b5006e8204e2c4"}
Oct 30 18:56:22.047536 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.047508    4544 scope.go:122] "RemoveContainer" containerID="48634614b12a7446f65c2d0af746b484963bf34fd8daba368a44839525984fe3"
Oct 30 18:56:22.059942 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.059924    4544 scope.go:122] "RemoveContainer" containerID="48634614b12a7446f65c2d0af746b484963bf34fd8daba368a44839525984fe3"
Oct 30 18:56:22.060212 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:56:22.060193    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48634614b12a7446f65c2d0af746b484963bf34fd8daba368a44839525984fe3\": container with ID starting with 48634614b12a7446f65c2d0af746b484963bf34fd8daba368a44839525984fe3 not found: ID does not exist" containerID="48634614b12a7446f65c2d0af746b484963bf34fd8daba368a44839525984fe3"
Oct 30 18:56:22.060285 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.060219    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48634614b12a7446f65c2d0af746b484963bf34fd8daba368a44839525984fe3"} err="failed to get container status \"48634614b12a7446f65c2d0af746b484963bf34fd8daba368a44839525984fe3\": rpc error: code = NotFound desc = could not find container \"48634614b12a7446f65c2d0af746b484963bf34fd8daba368a44839525984fe3\": container with ID starting with 48634614b12a7446f65c2d0af746b484963bf34fd8daba368a44839525984fe3 not found: ID does not exist"
Oct 30 18:56:22.203943 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.203855    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"]
Oct 30 18:56:22.207530 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.207491    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-fxssw"]
Oct 30 18:56:22.603183 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.603146    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2aaac2d4-38ca-46d5-95ea-52356e01647f" path="/var/lib/kubelet/pods/2aaac2d4-38ca-46d5-95ea-52356e01647f/volumes"
Oct 30 18:56:22.668643 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.668615    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-t7mdf-jdnm7"]
Oct 30 18:56:22.669086 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.669071    4544 cpu_manager.go:414] "containerMap: removing container" podUID="2aaac2d4-38ca-46d5-95ea-52356e01647f" containerName="istio-proxy"
Oct 30 18:56:22.669086 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.669086    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2aaac2d4-38ca-46d5-95ea-52356e01647f" containerName="istio-proxy"
Oct 30 18:56:22.669214 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.669209    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="2aaac2d4-38ca-46d5-95ea-52356e01647f" containerName="istio-proxy"
Oct 30 18:56:22.673812 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.673795    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-t7mdf-jdnm7"
Oct 30 18:56:22.723714 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.723692    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 18:56:22.723829 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.723809    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-xg84v\""
Oct 30 18:56:22.725384 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.725365    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-t7mdf-jdnm7"]
Oct 30 18:56:22.770544 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.770514    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/0c622da8-005a-499d-b9c3-e78275c370a2-archives-path\") pod \"periodic-gathering-t7mdf-jdnm7\" (UID: \"0c622da8-005a-499d-b9c3-e78275c370a2\") " pod="openshift-insights/periodic-gathering-t7mdf-jdnm7"
Oct 30 18:56:22.770682 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.770561    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh2lb\" (UniqueName: \"kubernetes.io/projected/0c622da8-005a-499d-b9c3-e78275c370a2-kube-api-access-qh2lb\") pod \"periodic-gathering-t7mdf-jdnm7\" (UID: \"0c622da8-005a-499d-b9c3-e78275c370a2\") " pod="openshift-insights/periodic-gathering-t7mdf-jdnm7"
Oct 30 18:56:22.770764 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.770743    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c622da8-005a-499d-b9c3-e78275c370a2-trusted-ca-bundle\") pod \"periodic-gathering-t7mdf-jdnm7\" (UID: \"0c622da8-005a-499d-b9c3-e78275c370a2\") " pod="openshift-insights/periodic-gathering-t7mdf-jdnm7"
Oct 30 18:56:22.770810 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.770794    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c622da8-005a-499d-b9c3-e78275c370a2-service-ca-bundle\") pod \"periodic-gathering-t7mdf-jdnm7\" (UID: \"0c622da8-005a-499d-b9c3-e78275c370a2\") " pod="openshift-insights/periodic-gathering-t7mdf-jdnm7"
Oct 30 18:56:22.816991 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.816961    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 18:56:22.872111 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.872050    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c622da8-005a-499d-b9c3-e78275c370a2-trusted-ca-bundle\") pod \"periodic-gathering-t7mdf-jdnm7\" (UID: \"0c622da8-005a-499d-b9c3-e78275c370a2\") " pod="openshift-insights/periodic-gathering-t7mdf-jdnm7"
Oct 30 18:56:22.872111 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.872094    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c622da8-005a-499d-b9c3-e78275c370a2-service-ca-bundle\") pod \"periodic-gathering-t7mdf-jdnm7\" (UID: \"0c622da8-005a-499d-b9c3-e78275c370a2\") " pod="openshift-insights/periodic-gathering-t7mdf-jdnm7"
Oct 30 18:56:22.872251 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.872139    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/0c622da8-005a-499d-b9c3-e78275c370a2-archives-path\") pod \"periodic-gathering-t7mdf-jdnm7\" (UID: \"0c622da8-005a-499d-b9c3-e78275c370a2\") " pod="openshift-insights/periodic-gathering-t7mdf-jdnm7"
Oct 30 18:56:22.872251 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.872194    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh2lb\" (UniqueName: \"kubernetes.io/projected/0c622da8-005a-499d-b9c3-e78275c370a2-kube-api-access-qh2lb\") pod \"periodic-gathering-t7mdf-jdnm7\" (UID: \"0c622da8-005a-499d-b9c3-e78275c370a2\") " pod="openshift-insights/periodic-gathering-t7mdf-jdnm7"
Oct 30 18:56:22.872562 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.872542    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/0c622da8-005a-499d-b9c3-e78275c370a2-archives-path\") pod \"periodic-gathering-t7mdf-jdnm7\" (UID: \"0c622da8-005a-499d-b9c3-e78275c370a2\") " pod="openshift-insights/periodic-gathering-t7mdf-jdnm7"
Oct 30 18:56:22.872848 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.872832    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c622da8-005a-499d-b9c3-e78275c370a2-service-ca-bundle\") pod \"periodic-gathering-t7mdf-jdnm7\" (UID: \"0c622da8-005a-499d-b9c3-e78275c370a2\") " pod="openshift-insights/periodic-gathering-t7mdf-jdnm7"
Oct 30 18:56:22.872924 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.872876    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c622da8-005a-499d-b9c3-e78275c370a2-trusted-ca-bundle\") pod \"periodic-gathering-t7mdf-jdnm7\" (UID: \"0c622da8-005a-499d-b9c3-e78275c370a2\") " pod="openshift-insights/periodic-gathering-t7mdf-jdnm7"
Oct 30 18:56:22.926683 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.926656    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh2lb\" (UniqueName: \"kubernetes.io/projected/0c622da8-005a-499d-b9c3-e78275c370a2-kube-api-access-qh2lb\") pod \"periodic-gathering-t7mdf-jdnm7\" (UID: \"0c622da8-005a-499d-b9c3-e78275c370a2\") " pod="openshift-insights/periodic-gathering-t7mdf-jdnm7"
Oct 30 18:56:22.986078 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:22.986056    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-t7mdf-jdnm7"
Oct 30 18:56:23.353298 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:23.353252    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-t7mdf-jdnm7"]
Oct 30 18:56:23.355792 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:56:23.355764    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c622da8_005a_499d_b9c3_e78275c370a2.slice/crio-112c54a077744d4aa48151b2d28ec4c3b62d19f749f9d60d3a57a366f18ad61c WatchSource:0}: Error finding container 112c54a077744d4aa48151b2d28ec4c3b62d19f749f9d60d3a57a366f18ad61c: Status 404 returned error can't find the container with id 112c54a077744d4aa48151b2d28ec4c3b62d19f749f9d60d3a57a366f18ad61c
Oct 30 18:56:24.065089 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:24.065051    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-t7mdf-jdnm7" event={"ID":"0c622da8-005a-499d-b9c3-e78275c370a2","Type":"ContainerStarted","Data":"d51c697aa18901d2d66e99ab6baa9bdf9cca839384d8abe4a159f5e233cb1502"}
Oct 30 18:56:24.067362 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:24.065103    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-t7mdf-jdnm7" event={"ID":"0c622da8-005a-499d-b9c3-e78275c370a2","Type":"ContainerStarted","Data":"112c54a077744d4aa48151b2d28ec4c3b62d19f749f9d60d3a57a366f18ad61c"}
Oct 30 18:56:24.173573 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:24.173519    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-t7mdf-jdnm7" podStartSLOduration=2.173504842 podStartE2EDuration="2.173504842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:24.171264485 +0000 UTC m=+1181.697139581" watchObservedRunningTime="2026-10-30 18:56:24.173504842 +0000 UTC m=+1181.699379937"
Oct 30 18:56:35.540787 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:56:35.540745    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:56:40.250509 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.250481    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"]
Oct 30 18:56:40.254536 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.254515    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.366454 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.366419    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"]
Oct 30 18:56:40.369553 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.369524    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-workload-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.369738 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.369660    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-credential-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.369738 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.369694    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ccbm\" (UniqueName: \"kubernetes.io/projected/30db7451-7fa3-443d-bfe9-41412759e3d8-kube-api-access-4ccbm\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.369738 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.369732    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-envoy\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.369922 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.369760    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-workload-certs\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.369922 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.369798    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30db7451-7fa3-443d-bfe9-41412759e3d8-istiod-ca-cert\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.370030 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.369922    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-data\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.370030 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.369949    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-token\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.370030 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.369991    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-podinfo\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.471356 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.471325    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-workload-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.471533 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.471362    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-credential-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.471533 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.471384    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ccbm\" (UniqueName: \"kubernetes.io/projected/30db7451-7fa3-443d-bfe9-41412759e3d8-kube-api-access-4ccbm\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.471533 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.471422    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-envoy\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.471533 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.471441    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-workload-certs\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.471533 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.471472    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30db7451-7fa3-443d-bfe9-41412759e3d8-istiod-ca-cert\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.471833 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.471617    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-data\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.471833 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.471643    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-token\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.471833 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.471685    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-podinfo\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.471833 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.471765    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-workload-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.472055 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.471834    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-credential-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.472055 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.471885    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-workload-certs\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.472055 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.471963    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-data\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.472242 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.472222    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30db7451-7fa3-443d-bfe9-41412759e3d8-istiod-ca-cert\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.473983 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.473962    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-envoy\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.474176 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.474156    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-podinfo\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.525248 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.525212    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-token\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.525382 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.525327    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ccbm\" (UniqueName: \"kubernetes.io/projected/30db7451-7fa3-443d-bfe9-41412759e3d8-kube-api-access-4ccbm\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.568455 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.568423    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:40.996591 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:40.996564    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"]
Oct 30 18:56:40.998583 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:56:40.998555    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30db7451_7fa3_443d_bfe9_41412759e3d8.slice/crio-5e3c7418608bfcad6946cfed32b90be0f6462e37ac85f9f94c9d2c738f479f0e WatchSource:0}: Error finding container 5e3c7418608bfcad6946cfed32b90be0f6462e37ac85f9f94c9d2c738f479f0e: Status 404 returned error can't find the container with id 5e3c7418608bfcad6946cfed32b90be0f6462e37ac85f9f94c9d2c738f479f0e
Oct 30 18:56:41.001175 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:41.001139    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:56:41.001282 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:41.001225    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:56:41.001282 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:41.001271    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:56:41.166460 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:41.166421    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc" event={"ID":"30db7451-7fa3-443d-bfe9-41412759e3d8","Type":"ContainerStarted","Data":"2e56fd1f367107e02e86487733010f26284de4c5d3de49f6bd3394b78308e9cb"}
Oct 30 18:56:41.166693 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:41.166494    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc" event={"ID":"30db7451-7fa3-443d-bfe9-41412759e3d8","Type":"ContainerStarted","Data":"5e3c7418608bfcad6946cfed32b90be0f6462e37ac85f9f94c9d2c738f479f0e"}
Oct 30 18:56:41.278499 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:41.278445    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc" podStartSLOduration=1.278429727 podStartE2EDuration="1.278429727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:41.275988304 +0000 UTC m=+1198.801863403" watchObservedRunningTime="2026-10-30 18:56:41.278429727 +0000 UTC m=+1198.804304823"
Oct 30 18:56:41.573687 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:41.568710    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:42.579627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:42.572777    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 18:56:42.579627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:42.572844    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc" podUID="30db7451-7fa3-443d-bfe9-41412759e3d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 18:56:42.734147 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:42.734118    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 18:56:42.737404 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:42.737383    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 18:56:43.573029 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:43.572990    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 18:56:43.573331 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:43.573300    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc" podUID="30db7451-7fa3-443d-bfe9-41412759e3d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 18:56:44.569720 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:44.569684    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 18:56:44.570325 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:44.570295    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc" podUID="30db7451-7fa3-443d-bfe9-41412759e3d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 18:56:45.574628 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:45.570082    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 18:56:45.574628 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:45.570153    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc" podUID="30db7451-7fa3-443d-bfe9-41412759e3d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 18:56:46.579387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:46.578954    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 18:56:46.579387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:46.579021    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc" podUID="30db7451-7fa3-443d-bfe9-41412759e3d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 18:56:47.569230 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:47.569187    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 18:56:47.569554 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:47.569257    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc" podUID="30db7451-7fa3-443d-bfe9-41412759e3d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 18:56:48.574646 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:48.574587    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:48.575065 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:48.575042    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:48.575752 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:48.575730    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 18:56:55.287650 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.287544    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"]
Oct 30 18:56:55.293354 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.293334    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.392764 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.392734    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"]
Oct 30 18:56:55.415987 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.415954    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f23bf845-e578-4ca7-bc75-898f940fc494-istio-token\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.416132 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.415999    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-istio-envoy\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.416208 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.416134    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f23bf845-e578-4ca7-bc75-898f940fc494-istio-podinfo\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.416208 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.416191    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-workload-socket\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.416337 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.416320    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-workload-certs\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.416387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.416362    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-credential-socket\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.416540 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.416514    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kjb5\" (UniqueName: \"kubernetes.io/projected/f23bf845-e578-4ca7-bc75-898f940fc494-kube-api-access-7kjb5\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.416688 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.416666    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-istio-data\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.416756 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.416707    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f23bf845-e578-4ca7-bc75-898f940fc494-istiod-ca-cert\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.518286 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.518248    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kjb5\" (UniqueName: \"kubernetes.io/projected/f23bf845-e578-4ca7-bc75-898f940fc494-kube-api-access-7kjb5\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.518477 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.518337    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-istio-data\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.518477 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.518368    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f23bf845-e578-4ca7-bc75-898f940fc494-istiod-ca-cert\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.518477 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.518430    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f23bf845-e578-4ca7-bc75-898f940fc494-istio-token\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.518477 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.518464    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-istio-envoy\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.518745 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.518531    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f23bf845-e578-4ca7-bc75-898f940fc494-istio-podinfo\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.518745 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.518570    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-workload-socket\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.518745 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.518657    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-workload-certs\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.518745 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.518694    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-credential-socket\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.518964 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.518738    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-istio-data\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.519022 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.519009    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-workload-socket\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.519150 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.519129    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-credential-socket\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.519205 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.519189    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f23bf845-e578-4ca7-bc75-898f940fc494-istiod-ca-cert\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.519244 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.519209    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-workload-certs\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.520662 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.520640    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-istio-envoy\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.521337 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.521322    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f23bf845-e578-4ca7-bc75-898f940fc494-istio-podinfo\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.572774 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.572706    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f23bf845-e578-4ca7-bc75-898f940fc494-istio-token\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.572901 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.572819    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kjb5\" (UniqueName: \"kubernetes.io/projected/f23bf845-e578-4ca7-bc75-898f940fc494-kube-api-access-7kjb5\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:55.606401 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:55.606373    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:56.043586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:56.043550    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"]
Oct 30 18:56:56.081160 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:56:56.081130    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf23bf845_e578_4ca7_bc75_898f940fc494.slice/crio-8c26acdec99ecb6dcec4ea1d03abfa4a61c5d85a824a60e354783b7be315300d WatchSource:0}: Error finding container 8c26acdec99ecb6dcec4ea1d03abfa4a61c5d85a824a60e354783b7be315300d: Status 404 returned error can't find the container with id 8c26acdec99ecb6dcec4ea1d03abfa4a61c5d85a824a60e354783b7be315300d
Oct 30 18:56:56.084193 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:56.084158    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:56:56.084286 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:56.084241    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:56:56.084286 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:56.084279    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:56:56.292838 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:56.292803    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6" event={"ID":"f23bf845-e578-4ca7-bc75-898f940fc494","Type":"ContainerStarted","Data":"a38f71934b5adc500da74fa0dfbc2fa57110255136717e478fffedb523d18c24"}
Oct 30 18:56:56.293247 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:56.292856    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6" event={"ID":"f23bf845-e578-4ca7-bc75-898f940fc494","Type":"ContainerStarted","Data":"8c26acdec99ecb6dcec4ea1d03abfa4a61c5d85a824a60e354783b7be315300d"}
Oct 30 18:56:56.403326 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:56.403214    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6" podStartSLOduration=1.403194762 podStartE2EDuration="1.403194762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:56.402273171 +0000 UTC m=+1213.928148282" watchObservedRunningTime="2026-10-30 18:56:56.403194762 +0000 UTC m=+1213.929069858"
Oct 30 18:56:56.606649 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:56.606620    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:56:57.608063 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:57.608030    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 30 18:56:57.608521 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:57.608087    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6" podUID="f23bf845-e578-4ca7-bc75-898f940fc494" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 30 18:56:58.607462 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:58.607429    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 30 18:56:58.607672 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:58.607482    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6" podUID="f23bf845-e578-4ca7-bc75-898f940fc494" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 30 18:56:59.612119 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:59.612077    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 30 18:56:59.612654 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:56:59.612161    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6" podUID="f23bf845-e578-4ca7-bc75-898f940fc494" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 30 18:57:00.607230 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:00.607192    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 30 18:57:00.607465 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:00.607249    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6" podUID="f23bf845-e578-4ca7-bc75-898f940fc494" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 30 18:57:01.606867 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:01.606831    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 30 18:57:01.607292 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:01.606890    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6" podUID="f23bf845-e578-4ca7-bc75-898f940fc494" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 30 18:57:02.609593 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:02.609542    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 30 18:57:02.610096 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:02.609643    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6" podUID="f23bf845-e578-4ca7-bc75-898f940fc494" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 30 18:57:03.611713 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:03.611675    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:57:03.612104 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:03.612082    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:57:03.612869 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:03.612848    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 18:57:05.532767 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:57:05.532733    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:57:10.266317 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.266284    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"]
Oct 30 18:57:10.270482 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.270465    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.313148 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.313119    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"]
Oct 30 18:57:10.316486 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.316468    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.337750 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.337724    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-istio-envoy\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.337908 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.337815    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-workload-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.337908 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.337875    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-workload-certs\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.338074 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.337984    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c22154c8-6939-496a-b422-29a775dcbf57-istiod-ca-cert\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.338074 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.338030    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c22154c8-6939-496a-b422-29a775dcbf57-istio-token\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.338213 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.338093    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-istio-data\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.338213 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.338155    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-credential-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.338328 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.338278    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx9n5\" (UniqueName: \"kubernetes.io/projected/c22154c8-6939-496a-b422-29a775dcbf57-kube-api-access-dx9n5\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.338666 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.338393    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c22154c8-6939-496a-b422-29a775dcbf57-istio-podinfo\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.375590 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.375559    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"]
Oct 30 18:57:10.406128 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.406097    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"]
Oct 30 18:57:10.439366 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.439336    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-istio-envoy\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.439534 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.439374    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-workload-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.439534 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.439395    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-data\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.439534 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.439430    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-workload-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.439534 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.439449    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-podinfo\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.439534 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.439473    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-workload-certs\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.439832 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.439532    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-token\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.439832 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.439734    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-credential-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.439832 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.439789    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c22154c8-6939-496a-b422-29a775dcbf57-istiod-ca-cert\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.439832 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.439826    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c22154c8-6939-496a-b422-29a775dcbf57-istio-token\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.440037 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.439855    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc47d\" (UniqueName: \"kubernetes.io/projected/13a222a4-d62a-4078-bd12-c03a8996dc87-kube-api-access-rc47d\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.440037 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.439878    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-workload-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.440037 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.439897    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-istio-data\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.440037 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.439938    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-workload-certs\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.440037 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.439943    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-credential-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.440037 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.440020    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-workload-certs\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.440343 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.440074    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx9n5\" (UniqueName: \"kubernetes.io/projected/c22154c8-6939-496a-b422-29a775dcbf57-kube-api-access-dx9n5\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.440343 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.440132    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c22154c8-6939-496a-b422-29a775dcbf57-istio-podinfo\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.440343 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.440201    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-credential-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.440343 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.440215    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13a222a4-d62a-4078-bd12-c03a8996dc87-istiod-ca-cert\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.440343 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.440314    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-envoy\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.440633 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.440373    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-istio-data\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.440633 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.440486    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c22154c8-6939-496a-b422-29a775dcbf57-istiod-ca-cert\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.441895 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.441872    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-istio-envoy\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.442505 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.442486    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c22154c8-6939-496a-b422-29a775dcbf57-istio-podinfo\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.495844 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.495819    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c22154c8-6939-496a-b422-29a775dcbf57-istio-token\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.496108 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.496086    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx9n5\" (UniqueName: \"kubernetes.io/projected/c22154c8-6939-496a-b422-29a775dcbf57-kube-api-access-dx9n5\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.541740 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.541661    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-token\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.541740 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.541723    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-credential-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.541929 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.541771    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc47d\" (UniqueName: \"kubernetes.io/projected/13a222a4-d62a-4078-bd12-c03a8996dc87-kube-api-access-rc47d\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.541929 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.541846    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-workload-certs\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.542035 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.541940    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13a222a4-d62a-4078-bd12-c03a8996dc87-istiod-ca-cert\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.542035 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.542009    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-envoy\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.542136 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.542086    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-data\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.542193 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.542138    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-workload-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.542193 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.542166    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-podinfo\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.542193 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.542169    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-credential-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.542344 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.542325    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-workload-certs\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.542530 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.542499    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-data\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.542694 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.542545    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-workload-socket\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.542799 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.542780    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13a222a4-d62a-4078-bd12-c03a8996dc87-istiod-ca-cert\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.544265 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.544236    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-envoy\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.544545 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.544526    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-podinfo\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.584150 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.584126    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:10.595278 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.595250    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-token\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.595904 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.595875    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc47d\" (UniqueName: \"kubernetes.io/projected/13a222a4-d62a-4078-bd12-c03a8996dc87-kube-api-access-rc47d\") pod \"e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") " pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:10.629586 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:10.629555    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:11.007489 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:11.007460    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"]
Oct 30 18:57:11.009446 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:57:11.009422    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc22154c8_6939_496a_b422_29a775dcbf57.slice/crio-41c8962fb7b7654be91160937c8f202e57d57507b2be25df3e34c75c50308cc9 WatchSource:0}: Error finding container 41c8962fb7b7654be91160937c8f202e57d57507b2be25df3e34c75c50308cc9: Status 404 returned error can't find the container with id 41c8962fb7b7654be91160937c8f202e57d57507b2be25df3e34c75c50308cc9
Oct 30 18:57:11.012231 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:11.012202    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:57:11.012305 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:11.012266    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:57:11.012341 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:11.012307    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:57:11.060914 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:11.060569    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"]
Oct 30 18:57:11.065866 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:57:11.065823    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13a222a4_d62a_4078_bd12_c03a8996dc87.slice/crio-fceffd05db4b38608f2788e21d0fce4fbacbbfcc729ec03044d3834f885983f7 WatchSource:0}: Error finding container fceffd05db4b38608f2788e21d0fce4fbacbbfcc729ec03044d3834f885983f7: Status 404 returned error can't find the container with id fceffd05db4b38608f2788e21d0fce4fbacbbfcc729ec03044d3834f885983f7
Oct 30 18:57:11.068913 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:11.068876    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:57:11.068990 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:11.068961    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:57:11.069034 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:11.069000    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:57:11.397863 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:11.397813    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj" event={"ID":"c22154c8-6939-496a-b422-29a775dcbf57","Type":"ContainerStarted","Data":"4b0cf9aafbfea3e2d0f7dfa6069b60bc0882fef0561e3c4d6d7ccb0fe88d6193"}
Oct 30 18:57:11.398276 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:11.397922    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj" event={"ID":"c22154c8-6939-496a-b422-29a775dcbf57","Type":"ContainerStarted","Data":"41c8962fb7b7654be91160937c8f202e57d57507b2be25df3e34c75c50308cc9"}
Oct 30 18:57:11.399853 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:11.399820    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk" event={"ID":"13a222a4-d62a-4078-bd12-c03a8996dc87","Type":"ContainerStarted","Data":"7ef2e7870d56ed94d0e6a364395f6fc30150b3fbc766d9c9ed18df9cd7f1737e"}
Oct 30 18:57:11.399990 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:11.399872    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk" event={"ID":"13a222a4-d62a-4078-bd12-c03a8996dc87","Type":"ContainerStarted","Data":"fceffd05db4b38608f2788e21d0fce4fbacbbfcc729ec03044d3834f885983f7"}
Oct 30 18:57:11.584750 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:11.584708    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:11.623285 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:11.623219    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk" podStartSLOduration=1.623202265 podStartE2EDuration="1.623202265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:11.622090182 +0000 UTC m=+1229.147965279" watchObservedRunningTime="2026-10-30 18:57:11.623202265 +0000 UTC m=+1229.149077362"
Oct 30 18:57:11.629722 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:11.629672    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:11.633377 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:11.633324    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj" podStartSLOduration=1.633308861 podStartE2EDuration="1.633308861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:11.508350715 +0000 UTC m=+1229.034225811" watchObservedRunningTime="2026-10-30 18:57:11.633308861 +0000 UTC m=+1229.159184018"
Oct 30 18:57:12.626630 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:12.623751    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:57:12.626630 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:12.623813    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj" podUID="c22154c8-6939-496a-b422-29a775dcbf57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 18:57:12.630923 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:12.630844    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:57:12.630923 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:12.630885    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk" podUID="13a222a4-d62a-4078-bd12-c03a8996dc87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 18:57:13.585391 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:13.584946    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:57:13.585391 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:13.585021    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj" podUID="c22154c8-6939-496a-b422-29a775dcbf57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 18:57:13.629914 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:13.629881    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:57:13.630482 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:13.630451    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk" podUID="13a222a4-d62a-4078-bd12-c03a8996dc87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 18:57:14.585003 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:14.584965    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:57:14.585988 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:14.585957    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj" podUID="c22154c8-6939-496a-b422-29a775dcbf57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 18:57:14.630437 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:14.630400    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:57:14.630824 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:14.630462    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk" podUID="13a222a4-d62a-4078-bd12-c03a8996dc87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 18:57:15.584630 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:15.584487    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:57:15.584630 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:15.584551    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj" podUID="c22154c8-6939-496a-b422-29a775dcbf57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 18:57:15.630251 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:15.630213    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:57:15.630424 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:15.630263    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk" podUID="13a222a4-d62a-4078-bd12-c03a8996dc87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 18:57:16.585705 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:16.585663    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:57:16.586407 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:16.586371    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj" podUID="c22154c8-6939-496a-b422-29a775dcbf57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 18:57:16.629926 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:16.629893    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:57:16.630089 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:16.629973    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk" podUID="13a222a4-d62a-4078-bd12-c03a8996dc87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 18:57:17.585031 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:17.584991    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:57:17.585263 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:17.585067    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj" podUID="c22154c8-6939-496a-b422-29a775dcbf57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 18:57:17.630151 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:17.630115    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:57:17.630677 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:17.630169    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk" podUID="13a222a4-d62a-4078-bd12-c03a8996dc87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 18:57:18.690153 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:18.690111    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:18.690794 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:18.690467    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:18.691549 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:18.691520    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 18:57:18.779770 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:18.779737    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:18.780165 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:18.780141    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:18.780991 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:18.780973    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 18:57:35.531660 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:57:35.531624    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:57:40.250629 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.250569    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"]
Oct 30 18:57:40.254736 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.254714    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.331270 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.331226    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"]
Oct 30 18:57:40.332985 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.332951    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-workload-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.333184 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.333117    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-envoy\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.333184 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.333157    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-workload-certs\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.333296 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.333274    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istiod-ca-cert\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.333353 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.333330    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-token\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.333405 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.333374    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-podinfo\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.333466 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.333449    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95ktm\" (UniqueName: \"kubernetes.io/projected/49a8e4ba-b424-4727-8e8b-23a3eaccece2-kube-api-access-95ktm\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.333578 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.333552    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-data\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.333729 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.333618    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-credential-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.434836 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.434803    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95ktm\" (UniqueName: \"kubernetes.io/projected/49a8e4ba-b424-4727-8e8b-23a3eaccece2-kube-api-access-95ktm\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.435010 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.434887    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-data\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.435010 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.434925    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-credential-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.435122 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.435068    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-workload-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.435219 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.435190    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-envoy\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.435277 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.435244    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-workload-certs\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.435336 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.435292    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istiod-ca-cert\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.435336 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.435307    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-credential-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.435336 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.435325    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-token\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.435487 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.435345    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-data\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.435487 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.435359    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-podinfo\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.435487 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.435429    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-workload-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.435674 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.435643    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-workload-certs\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.435957 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.435930    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istiod-ca-cert\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.437375 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.437354    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-envoy\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.437706 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.437689    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-podinfo\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.489178 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.489144    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-token\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.489330 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.489316    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95ktm\" (UniqueName: \"kubernetes.io/projected/49a8e4ba-b424-4727-8e8b-23a3eaccece2-kube-api-access-95ktm\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:40.569994 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:40.569952    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:41.002621 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:41.002577    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"]
Oct 30 18:57:41.003696 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:57:41.003659    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49a8e4ba_b424_4727_8e8b_23a3eaccece2.slice/crio-56465488a396008e47af3cffa2f86569f8171ee74f644935482bd590af47f7fc WatchSource:0}: Error finding container 56465488a396008e47af3cffa2f86569f8171ee74f644935482bd590af47f7fc: Status 404 returned error can't find the container with id 56465488a396008e47af3cffa2f86569f8171ee74f644935482bd590af47f7fc
Oct 30 18:57:41.006282 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:41.006247    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:57:41.006396 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:41.006330    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:57:41.006396 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:41.006373    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:57:41.602867 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:41.602833    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff" event={"ID":"49a8e4ba-b424-4727-8e8b-23a3eaccece2","Type":"ContainerStarted","Data":"72f098bcdf198672d76b3899149ab9fbe84db3b45d641040b406a2e5cc6133dc"}
Oct 30 18:57:41.603420 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:41.602881    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff" event={"ID":"49a8e4ba-b424-4727-8e8b-23a3eaccece2","Type":"ContainerStarted","Data":"56465488a396008e47af3cffa2f86569f8171ee74f644935482bd590af47f7fc"}
Oct 30 18:57:41.826096 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:41.825984    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff" podStartSLOduration=1.8259657900000001 podStartE2EDuration="1.82596579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:41.779474467 +0000 UTC m=+1259.305349689" watchObservedRunningTime="2026-10-30 18:57:41.82596579 +0000 UTC m=+1259.351840889"
Oct 30 18:57:42.577698 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:42.577662    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:42.582005 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:42.581700    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 18:57:42.582005 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:42.581756    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff" podUID="49a8e4ba-b424-4727-8e8b-23a3eaccece2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 18:57:43.573978 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:43.573929    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 18:57:43.574563 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:43.574517    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff" podUID="49a8e4ba-b424-4727-8e8b-23a3eaccece2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 18:57:44.582014 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:44.581976    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 18:57:44.582595 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:44.582564    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff" podUID="49a8e4ba-b424-4727-8e8b-23a3eaccece2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 18:57:45.572034 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:45.572000    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 18:57:45.572337 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:45.572292    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff" podUID="49a8e4ba-b424-4727-8e8b-23a3eaccece2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 18:57:46.580629 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:46.573960    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 18:57:46.580629 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:46.574021    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff" podUID="49a8e4ba-b424-4727-8e8b-23a3eaccece2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 18:57:47.573984 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:47.573946    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 18:57:47.574290 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:47.574260    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff" podUID="49a8e4ba-b424-4727-8e8b-23a3eaccece2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 18:57:48.810343 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:48.810305    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:48.824160 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:48.824130    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:48.855989 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:48.855958    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 18:57:49.761626 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:49.760095    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:57:49.761626 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:49.760165    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj" podUID="c22154c8-6939-496a-b422-29a775dcbf57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:57:49.799120 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:49.799083    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:57:49.799407 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:49.799377    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7" podUID="48229a00-0acf-4c2b-b14e-d0151692b2d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:57:49.948740 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:49.948693    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:57:49.974207 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:49.969682    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:57:59.819531 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:59.819499    4544 generic.go:356] "Generic (PLEG): container finished" podID="0c622da8-005a-499d-b9c3-e78275c370a2" containerID="d51c697aa18901d2d66e99ab6baa9bdf9cca839384d8abe4a159f5e233cb1502" exitCode=0
Oct 30 18:57:59.819950 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:57:59.819578    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-t7mdf-jdnm7" event={"ID":"0c622da8-005a-499d-b9c3-e78275c370a2","Type":"ContainerDied","Data":"d51c697aa18901d2d66e99ab6baa9bdf9cca839384d8abe4a159f5e233cb1502"}
Oct 30 18:58:00.176777 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.176695    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"]
Oct 30 18:58:00.180753 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.180733    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.231222 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.231175    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-75e91b\"/\"distinct-2-w4-75e91b-kserve-self-signed-certs\""
Oct 30 18:58:00.235540 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.235500    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"]
Oct 30 18:58:00.293195 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.293164    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-home\") pod \"distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.293387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.293203    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dqvv\" (UniqueName: \"kubernetes.io/projected/fd9777c9-1b40-42fd-9815-cdf1a3509661-kube-api-access-4dqvv\") pod \"distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.293387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.293244    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-model-cache\") pod \"distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.293387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.293351    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd9777c9-1b40-42fd-9815-cdf1a3509661-tls-certs\") pod \"distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.293387 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.293386    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-dshm\") pod \"distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.293640 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.293411    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-tmp-dir\") pod \"distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.394617 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.394578    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-dshm\") pod \"distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.394787 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.394634    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-tmp-dir\") pod \"distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.394787 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.394681    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-home\") pod \"distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.394787 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.394700    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dqvv\" (UniqueName: \"kubernetes.io/projected/fd9777c9-1b40-42fd-9815-cdf1a3509661-kube-api-access-4dqvv\") pod \"distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.394964 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.394845    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-model-cache\") pod \"distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.395079 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.395057    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd9777c9-1b40-42fd-9815-cdf1a3509661-tls-certs\") pod \"distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.395219 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.395145    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-home\") pod \"distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.395219 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.395166    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-tmp-dir\") pod \"distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.395346 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.395298    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-model-cache\") pod \"distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.397388 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.397364    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-dshm\") pod \"distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.397492 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.397466    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd9777c9-1b40-42fd-9815-cdf1a3509661-tls-certs\") pod \"distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.452162 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.452100    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dqvv\" (UniqueName: \"kubernetes.io/projected/fd9777c9-1b40-42fd-9815-cdf1a3509661-kube-api-access-4dqvv\") pod \"distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") " pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.494370 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.494335    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:00.889790 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.889756    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"]
Oct 30 18:58:00.899975 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:58:00.899943    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd9777c9_1b40_42fd_9815_cdf1a3509661.slice/crio-fa6904fbbdb956b740ff19adbf5aaaafbe84ffada89362d45231afeb8cf44e90 WatchSource:0}: Error finding container fa6904fbbdb956b740ff19adbf5aaaafbe84ffada89362d45231afeb8cf44e90: Status 404 returned error can't find the container with id fa6904fbbdb956b740ff19adbf5aaaafbe84ffada89362d45231afeb8cf44e90
Oct 30 18:58:00.976278 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:00.976253    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-t7mdf-jdnm7"
Oct 30 18:58:01.126999 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:01.126914    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c622da8-005a-499d-b9c3-e78275c370a2-kube-api-access-qh2lb\" (UniqueName: \"kubernetes.io/projected/0c622da8-005a-499d-b9c3-e78275c370a2-kube-api-access-qh2lb\") pod \"0c622da8-005a-499d-b9c3-e78275c370a2\" (UID: \"0c622da8-005a-499d-b9c3-e78275c370a2\") "
Oct 30 18:58:01.126999 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:01.126974    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c622da8-005a-499d-b9c3-e78275c370a2-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/0c622da8-005a-499d-b9c3-e78275c370a2-archives-path\") pod \"0c622da8-005a-499d-b9c3-e78275c370a2\" (UID: \"0c622da8-005a-499d-b9c3-e78275c370a2\") "
Oct 30 18:58:01.127233 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:01.127075    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c622da8-005a-499d-b9c3-e78275c370a2-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c622da8-005a-499d-b9c3-e78275c370a2-trusted-ca-bundle\") pod \"0c622da8-005a-499d-b9c3-e78275c370a2\" (UID: \"0c622da8-005a-499d-b9c3-e78275c370a2\") "
Oct 30 18:58:01.127233 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:01.127107    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c622da8-005a-499d-b9c3-e78275c370a2-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c622da8-005a-499d-b9c3-e78275c370a2-service-ca-bundle\") pod \"0c622da8-005a-499d-b9c3-e78275c370a2\" (UID: \"0c622da8-005a-499d-b9c3-e78275c370a2\") "
Oct 30 18:58:01.127346 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:01.127317    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c622da8-005a-499d-b9c3-e78275c370a2-archives-path" pod "0c622da8-005a-499d-b9c3-e78275c370a2" (UID: "0c622da8-005a-499d-b9c3-e78275c370a2"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:01.127466 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:01.127442    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c622da8-005a-499d-b9c3-e78275c370a2-trusted-ca-bundle" pod "0c622da8-005a-499d-b9c3-e78275c370a2" (UID: "0c622da8-005a-499d-b9c3-e78275c370a2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:01.127515 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:01.127478    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c622da8-005a-499d-b9c3-e78275c370a2-service-ca-bundle" pod "0c622da8-005a-499d-b9c3-e78275c370a2" (UID: "0c622da8-005a-499d-b9c3-e78275c370a2"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:01.127782 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:01.127759    4544 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/0c622da8-005a-499d-b9c3-e78275c370a2-archives-path\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:01.127782 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:01.127782    4544 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c622da8-005a-499d-b9c3-e78275c370a2-trusted-ca-bundle\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:01.127924 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:01.127795    4544 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c622da8-005a-499d-b9c3-e78275c370a2-service-ca-bundle\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:01.129376 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:01.129357    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c622da8-005a-499d-b9c3-e78275c370a2-kube-api-access-qh2lb" pod "0c622da8-005a-499d-b9c3-e78275c370a2" (UID: "0c622da8-005a-499d-b9c3-e78275c370a2"). InnerVolumeSpecName "kube-api-access-qh2lb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:01.229427 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:01.229393    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh2lb\" (UniqueName: \"kubernetes.io/projected/0c622da8-005a-499d-b9c3-e78275c370a2-kube-api-access-qh2lb\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:01.831779 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:01.831751    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-t7mdf-jdnm7"
Oct 30 18:58:01.831957 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:01.831754    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-t7mdf-jdnm7" event={"ID":"0c622da8-005a-499d-b9c3-e78275c370a2","Type":"ContainerDied","Data":"112c54a077744d4aa48151b2d28ec4c3b62d19f749f9d60d3a57a366f18ad61c"}
Oct 30 18:58:01.831957 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:01.831883    4544 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="112c54a077744d4aa48151b2d28ec4c3b62d19f749f9d60d3a57a366f18ad61c"
Oct 30 18:58:01.833548 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:01.833519    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" event={"ID":"fd9777c9-1b40-42fd-9815-cdf1a3509661","Type":"ContainerStarted","Data":"65ed675a154c00dc393e32f838aed409a225c7d1175dc8d8f7684f27654de8f3"}
Oct 30 18:58:01.833703 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:01.833562    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" event={"ID":"fd9777c9-1b40-42fd-9815-cdf1a3509661","Type":"ContainerStarted","Data":"fa6904fbbdb956b740ff19adbf5aaaafbe84ffada89362d45231afeb8cf44e90"}
Oct 30 18:58:01.994656 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:01.991248    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podStartSLOduration=1.991227866 podStartE2EDuration="1.991227866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:01.951806325 +0000 UTC m=+1279.477681421" watchObservedRunningTime="2026-10-30 18:58:01.991227866 +0000 UTC m=+1279.517102963"
Oct 30 18:58:02.495026 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:02.494992    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:05.532741 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:58:05.532702    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:58:10.495307 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:10.495270    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:10.508481 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:10.508445    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:10.900105 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:10.900073    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:58:24.090523 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.090429    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"]
Oct 30 18:58:24.091113 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.091092    4544 cpu_manager.go:414] "containerMap: removing container" podUID="0c622da8-005a-499d-b9c3-e78275c370a2" containerName="insights-gathering"
Oct 30 18:58:24.091205 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.091115    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c622da8-005a-499d-b9c3-e78275c370a2" containerName="insights-gathering"
Oct 30 18:58:24.091256 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.091225    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c622da8-005a-499d-b9c3-e78275c370a2" containerName="insights-gathering"
Oct 30 18:58:24.096367 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.096347    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.162798 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.162768    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f5705f\"/\"distinct-2-w0-f5705f-kserve-self-signed-certs\""
Oct 30 18:58:24.212624 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.212578    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"]
Oct 30 18:58:24.237565 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.237521    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-model-cache\") pod \"distinct-2-w0-f5705f-kserve-747d77cf-8d4fb\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") " pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.237565 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.237582    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqcvs\" (UniqueName: \"kubernetes.io/projected/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-kube-api-access-wqcvs\") pod \"distinct-2-w0-f5705f-kserve-747d77cf-8d4fb\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") " pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.237966 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.237666    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-dshm\") pod \"distinct-2-w0-f5705f-kserve-747d77cf-8d4fb\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") " pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.237966 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.237734    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-tmp-dir\") pod \"distinct-2-w0-f5705f-kserve-747d77cf-8d4fb\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") " pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.237966 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.237776    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-tls-certs\") pod \"distinct-2-w0-f5705f-kserve-747d77cf-8d4fb\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") " pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.237966 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.237934    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-home\") pod \"distinct-2-w0-f5705f-kserve-747d77cf-8d4fb\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") " pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.339732 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.339694    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-model-cache\") pod \"distinct-2-w0-f5705f-kserve-747d77cf-8d4fb\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") " pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.339926 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.339748    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqcvs\" (UniqueName: \"kubernetes.io/projected/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-kube-api-access-wqcvs\") pod \"distinct-2-w0-f5705f-kserve-747d77cf-8d4fb\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") " pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.339926 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.339816    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-dshm\") pod \"distinct-2-w0-f5705f-kserve-747d77cf-8d4fb\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") " pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.339926 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.339883    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-tmp-dir\") pod \"distinct-2-w0-f5705f-kserve-747d77cf-8d4fb\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") " pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.340093 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.339926    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-tls-certs\") pod \"distinct-2-w0-f5705f-kserve-747d77cf-8d4fb\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") " pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.340158 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.340103    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-home\") pod \"distinct-2-w0-f5705f-kserve-747d77cf-8d4fb\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") " pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.340211 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.340160    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-model-cache\") pod \"distinct-2-w0-f5705f-kserve-747d77cf-8d4fb\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") " pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.340268 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.340235    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-tmp-dir\") pod \"distinct-2-w0-f5705f-kserve-747d77cf-8d4fb\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") " pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.340484 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.340457    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-home\") pod \"distinct-2-w0-f5705f-kserve-747d77cf-8d4fb\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") " pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.342246 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.342195    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-dshm\") pod \"distinct-2-w0-f5705f-kserve-747d77cf-8d4fb\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") " pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.342690 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.342671    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-tls-certs\") pod \"distinct-2-w0-f5705f-kserve-747d77cf-8d4fb\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") " pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.399935 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.399911    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqcvs\" (UniqueName: \"kubernetes.io/projected/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-kube-api-access-wqcvs\") pod \"distinct-2-w0-f5705f-kserve-747d77cf-8d4fb\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") " pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.411923 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.411904    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:24.796598 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.796568    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"]
Oct 30 18:58:24.797975 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:58:24.797946    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod91f8e092_eb1a_4c0f_9c1c_35195b4c8ed9.slice/crio-941030b1cfd17c1b0fc92503e739dbde9b48d41cc2531143bc2ad88a5f990cc4 WatchSource:0}: Error finding container 941030b1cfd17c1b0fc92503e739dbde9b48d41cc2531143bc2ad88a5f990cc4: Status 404 returned error can't find the container with id 941030b1cfd17c1b0fc92503e739dbde9b48d41cc2531143bc2ad88a5f990cc4
Oct 30 18:58:24.975940 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.975903    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" event={"ID":"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9","Type":"ContainerStarted","Data":"e25da3c1ffdc4287c16b1eae1abd6846f72686e98839a185866d7e601bf67620"}
Oct 30 18:58:24.975940 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:24.975945    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" event={"ID":"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9","Type":"ContainerStarted","Data":"941030b1cfd17c1b0fc92503e739dbde9b48d41cc2531143bc2ad88a5f990cc4"}
Oct 30 18:58:25.084397 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.084304    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podStartSLOduration=1.084290543 podStartE2EDuration="1.084290543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:25.0838793 +0000 UTC m=+1302.609754395" watchObservedRunningTime="2026-10-30 18:58:25.084290543 +0000 UTC m=+1302.610165642"
Oct 30 18:58:25.296082 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.296046    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"]
Oct 30 18:58:25.300853 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.300831    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.361977 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.361913    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"]
Oct 30 18:58:25.413071 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.413044    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:25.459249 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.459212    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-token\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.459503 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.459324    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rlbz\" (UniqueName: \"kubernetes.io/projected/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-kube-api-access-5rlbz\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.459503 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.459461    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-workload-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.459503 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.459489    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-envoy\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.459754 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.459517    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-credential-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.459754 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.459570    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-data\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.459754 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.459617    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-workload-certs\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.459754 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.459651    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istiod-ca-cert\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.459754 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.459712    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-podinfo\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.561282 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.561252    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-workload-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.561282 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.561286    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-envoy\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.562038 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.561314    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-credential-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.562038 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.561365    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-data\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.562038 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.561457    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-workload-certs\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.562038 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.561574    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istiod-ca-cert\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.562038 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.561809    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-workload-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.562038 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.561897    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-data\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.562038 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.561944    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-workload-certs\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.562466 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.562079    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-podinfo\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.562466 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.562145    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-token\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.562466 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.562247    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rlbz\" (UniqueName: \"kubernetes.io/projected/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-kube-api-access-5rlbz\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.562466 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.562080    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-credential-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.562466 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.562342    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istiod-ca-cert\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.563847 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.563818    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-envoy\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.564434 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.564413    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-podinfo\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.628857 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.628799    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-token\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.629025 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.629003    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rlbz\" (UniqueName: \"kubernetes.io/projected/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-kube-api-access-5rlbz\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:25.913948 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:25.913858    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:26.335376 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:26.335348    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"]
Oct 30 18:58:26.336192 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:58:26.336162    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8626c2ef_59e9_4fce_90a9_5eb28e6dfd4d.slice/crio-20c52d66c60d3752a88efc2d44e358ac862d7a041138ef897f346265ffaed654 WatchSource:0}: Error finding container 20c52d66c60d3752a88efc2d44e358ac862d7a041138ef897f346265ffaed654: Status 404 returned error can't find the container with id 20c52d66c60d3752a88efc2d44e358ac862d7a041138ef897f346265ffaed654
Oct 30 18:58:26.338750 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:26.338717    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:58:26.338850 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:26.338781    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:58:26.338850 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:26.338808    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:58:26.994137 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:26.994102    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt" event={"ID":"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d","Type":"ContainerStarted","Data":"c8034dc7a58213ddbeb20249d60b0445ce4218f1ed562113b4fd36b85249c4b9"}
Oct 30 18:58:26.994411 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:26.994379    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt" event={"ID":"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d","Type":"ContainerStarted","Data":"20c52d66c60d3752a88efc2d44e358ac862d7a041138ef897f346265ffaed654"}
Oct 30 18:58:27.201996 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:27.201946    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt" podStartSLOduration=2.201931948 podStartE2EDuration="2.201931948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:27.15548156 +0000 UTC m=+1304.681356659" watchObservedRunningTime="2026-10-30 18:58:27.201931948 +0000 UTC m=+1304.727807044"
Oct 30 18:58:27.914442 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:27.914407    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:27.916311 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:27.916279    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:58:27.916460 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:27.916330    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt" podUID="8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 18:58:28.915236 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:28.915184    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:58:28.915658 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:28.915250    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt" podUID="8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 18:58:29.914584 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:29.914551    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:58:29.914784 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:29.914614    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt" podUID="8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 18:58:30.914925 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:30.914891    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:58:30.915333 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:30.914941    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt" podUID="8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 18:58:31.915022 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:31.914985    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:58:31.915423 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:31.915039    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt" podUID="8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 18:58:32.914662 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:32.914591    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:58:32.914843 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:32.914690    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt" podUID="8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 30 18:58:33.919222 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:33.919189    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:33.919727 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:33.919574    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:33.920467 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:33.920444    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 18:58:34.412320 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:34.412287    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:34.425515 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:34.425491    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:35.056484 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:35.056458    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:58:35.539041 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:58:35.538999    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:58:40.321702 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.321670    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"]
Oct 30 18:58:40.325820 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.325800    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.434463 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.434423    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"]
Oct 30 18:58:40.483511 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.483468    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-credential-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.483729 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.483524    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-envoy\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.483729 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.483635    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60c1bed3-9d33-4979-815a-84a7acc09c38-istiod-ca-cert\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.483729 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.483692    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndwvg\" (UniqueName: \"kubernetes.io/projected/60c1bed3-9d33-4979-815a-84a7acc09c38-kube-api-access-ndwvg\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.483907 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.483782    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-podinfo\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.483907 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.483883    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-workload-certs\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.484006 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.483936    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-workload-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.484006 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.483959    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-data\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.484114 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.484051    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-token\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.585008 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.584910    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-credential-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.585008 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.584985    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-envoy\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.585261 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.585025    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60c1bed3-9d33-4979-815a-84a7acc09c38-istiod-ca-cert\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.585261 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.585088    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndwvg\" (UniqueName: \"kubernetes.io/projected/60c1bed3-9d33-4979-815a-84a7acc09c38-kube-api-access-ndwvg\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.585261 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.585142    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-podinfo\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.585261 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.585213    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-workload-certs\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.585472 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.585292    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-workload-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.585472 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.585332    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-data\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.585472 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.585341    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-credential-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.585472 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.585458    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-token\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.585713 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.585673    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-data\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.585814 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.585787    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60c1bed3-9d33-4979-815a-84a7acc09c38-istiod-ca-cert\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.585889 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.585820    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-workload-certs\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.585980 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.585960    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-workload-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.587998 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.587965    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-envoy\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.588461 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.588441    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-podinfo\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.639375 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.639349    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-token\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.639561 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.639535    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndwvg\" (UniqueName: \"kubernetes.io/projected/60c1bed3-9d33-4979-815a-84a7acc09c38-kube-api-access-ndwvg\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:40.939375 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:40.939305    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:41.378282 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:41.378252    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"]
Oct 30 18:58:41.380128 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:58:41.380104    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60c1bed3_9d33_4979_815a_84a7acc09c38.slice/crio-a8ed1051d7a1bc34db123bd2552b4ebd68e5f62136693a192a96efc9983edf77 WatchSource:0}: Error finding container a8ed1051d7a1bc34db123bd2552b4ebd68e5f62136693a192a96efc9983edf77: Status 404 returned error can't find the container with id a8ed1051d7a1bc34db123bd2552b4ebd68e5f62136693a192a96efc9983edf77
Oct 30 18:58:41.383207 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:41.383173    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:58:41.383300 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:41.383254    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:58:41.383300 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:41.383280    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:58:42.100320 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:42.100280    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6" event={"ID":"60c1bed3-9d33-4979-815a-84a7acc09c38","Type":"ContainerStarted","Data":"85574b987656d051cc5ebdcfdbec662c1baa5778a38a7d9280f1c2c3ecfb0dc6"}
Oct 30 18:58:42.100517 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:42.100347    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6" event={"ID":"60c1bed3-9d33-4979-815a-84a7acc09c38","Type":"ContainerStarted","Data":"a8ed1051d7a1bc34db123bd2552b4ebd68e5f62136693a192a96efc9983edf77"}
Oct 30 18:58:42.275862 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:42.275796    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6" podStartSLOduration=2.275773057 podStartE2EDuration="2.275773057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:42.261878798 +0000 UTC m=+1319.787753931" watchObservedRunningTime="2026-10-30 18:58:42.275773057 +0000 UTC m=+1319.801648154"
Oct 30 18:58:42.940177 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:42.940143    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:42.941520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:42.941483    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 18:58:42.941651 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:42.941528    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6" podUID="60c1bed3-9d33-4979-815a-84a7acc09c38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 18:58:43.940385 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:43.940346    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 18:58:43.940897 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:43.940405    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6" podUID="60c1bed3-9d33-4979-815a-84a7acc09c38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 18:58:44.939805 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:44.939767    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 18:58:44.939981 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:44.939823    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6" podUID="60c1bed3-9d33-4979-815a-84a7acc09c38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 18:58:45.940351 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:45.940318    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 18:58:45.940747 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:45.940373    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6" podUID="60c1bed3-9d33-4979-815a-84a7acc09c38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 18:58:46.939632 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:46.939575    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 18:58:46.939818 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:46.939653    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6" podUID="60c1bed3-9d33-4979-815a-84a7acc09c38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 18:58:47.940216 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:47.940181    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 18:58:47.940575 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:47.940268    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6" podUID="60c1bed3-9d33-4979-815a-84a7acc09c38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 18:58:48.944505 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:48.944470    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:48.944986 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:48.944945    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:48.946077 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:48.946056    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 18:58:55.463494 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.463455    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"]
Oct 30 18:58:55.469749 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.469728    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.509922 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.509887    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-istio-envoy\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.510300 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.510266    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-workload-socket\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.510471 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.510333    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/282d4bba-4530-4754-906f-f743b011ab66-istio-token\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.510471 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.510380    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-credential-socket\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.510471 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.510446    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/282d4bba-4530-4754-906f-f743b011ab66-istio-podinfo\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.510711 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.510485    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/282d4bba-4530-4754-906f-f743b011ab66-istiod-ca-cert\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.510711 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.510524    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9wbl\" (UniqueName: \"kubernetes.io/projected/282d4bba-4530-4754-906f-f743b011ab66-kube-api-access-p9wbl\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.510711 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.510639    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-istio-data\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.510711 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.510678    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-workload-certs\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.564565 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.564529    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"]
Oct 30 18:58:55.612500 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.612459    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-istio-data\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.612778 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.612757    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-workload-certs\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.612941 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.612926    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-istio-envoy\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.613110 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.613095    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-workload-socket\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.613235 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.613220    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/282d4bba-4530-4754-906f-f743b011ab66-istio-token\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.613361 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.613344    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-credential-socket\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.613513 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.613497    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/282d4bba-4530-4754-906f-f743b011ab66-istio-podinfo\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.613649 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.613633    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/282d4bba-4530-4754-906f-f743b011ab66-istiod-ca-cert\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.613786 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.613767    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9wbl\" (UniqueName: \"kubernetes.io/projected/282d4bba-4530-4754-906f-f743b011ab66-kube-api-access-p9wbl\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.615026 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.614928    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-credential-socket\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.615159 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.615028    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-workload-certs\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.615346 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.615301    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-workload-socket\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.615826 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.615635    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-istio-data\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.615826 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.615787    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/282d4bba-4530-4754-906f-f743b011ab66-istiod-ca-cert\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.617668 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.617648    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-istio-envoy\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.618540 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.618510    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/282d4bba-4530-4754-906f-f743b011ab66-istio-podinfo\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.668778 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.668743    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/282d4bba-4530-4754-906f-f743b011ab66-istio-token\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.668958 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.668928    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9wbl\" (UniqueName: \"kubernetes.io/projected/282d4bba-4530-4754-906f-f743b011ab66-kube-api-access-p9wbl\") pod \"e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") " pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:55.784859 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:55.784819    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:56.279388 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:56.279360    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"]
Oct 30 18:58:56.281019 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:58:56.280989    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod282d4bba_4530_4754_906f_f743b011ab66.slice/crio-fad63d922c71bb96634c38dc6dfef792c1f2ff421347309b1ba9cb87077ba42b WatchSource:0}: Error finding container fad63d922c71bb96634c38dc6dfef792c1f2ff421347309b1ba9cb87077ba42b: Status 404 returned error can't find the container with id fad63d922c71bb96634c38dc6dfef792c1f2ff421347309b1ba9cb87077ba42b
Oct 30 18:58:56.283892 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:56.283861    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:58:56.283973 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:56.283934    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:58:56.283973 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:56.283965    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:58:57.212549 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:57.212516    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw" event={"ID":"282d4bba-4530-4754-906f-f743b011ab66","Type":"ContainerStarted","Data":"bb4c3ad52dd097ccb283d270f5c51e9d338ccedee35a9f8a6bb5ab5997ca79f0"}
Oct 30 18:58:57.212980 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:57.212567    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw" event={"ID":"282d4bba-4530-4754-906f-f743b011ab66","Type":"ContainerStarted","Data":"fad63d922c71bb96634c38dc6dfef792c1f2ff421347309b1ba9cb87077ba42b"}
Oct 30 18:58:57.435553 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:57.435497    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw" podStartSLOduration=2.435480462 podStartE2EDuration="2.435480462s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:57.390241514 +0000 UTC m=+1334.916116613" watchObservedRunningTime="2026-10-30 18:58:57.435480462 +0000 UTC m=+1334.961355557"
Oct 30 18:58:57.786624 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:57.785981    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:58:57.793176 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:57.792735    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:58:57.793176 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:57.792842    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw" podUID="282d4bba-4530-4754-906f-f743b011ab66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 18:58:58.787019 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:58.786977    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:58:58.787686 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:58.787638    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw" podUID="282d4bba-4530-4754-906f-f743b011ab66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 18:58:59.785275 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:59.785233    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:58:59.785491 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:58:59.785314    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw" podUID="282d4bba-4530-4754-906f-f743b011ab66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 18:59:00.786021 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:00.785968    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:59:00.786628 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:00.786580    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw" podUID="282d4bba-4530-4754-906f-f743b011ab66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 18:59:01.805068 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:01.805028    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:59:01.805729 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:01.805681    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw" podUID="282d4bba-4530-4754-906f-f743b011ab66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 18:59:01.903641 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:01.899152    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 18:59:01.950726 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:01.950681    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 18:59:01.965768 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:01.965713    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:02.068161 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:02.068052    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:02.785521 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:02.785479    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:59:02.785733 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:02.785555    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw" podUID="282d4bba-4530-4754-906f-f743b011ab66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 18:59:04.219615 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:04.219575    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:59:04.220805 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:04.220779    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:59:04.221613 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:04.221582    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 18:59:05.532321 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:59:05.532285    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:59:20.504183 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:20.504141    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:20.590734 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:20.590683    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:20.718936 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:20.718886    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 18:59:20.908628 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:20.891707    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 18:59:20.957627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:20.956696    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:20.957627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:20.956800    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:21.508627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:21.507698    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/health\": context deadline exceeded"
Oct 30 18:59:21.508627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:21.508040    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:21.508627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:21.508448    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:59:21.607627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:21.606983    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:21.644902 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:21.644865    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:59:21.954419 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:21.954378    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:21.988917 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:21.956585    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:59:22.038632 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:22.025758    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 18:59:22.038632 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:22.026955    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:59:22.038632 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:22.036944    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 18:59:22.038632 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:22.037231    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 18:59:22.038632 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:22.037464    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:59:22.539625 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:22.512708    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 18:59:22.570688 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:22.570655    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:59:22.609695 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:22.609654    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:22.627628 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:22.610663    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 18:59:22.721545 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:22.684815    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:59:22.721545 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:22.684876    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk" podUID="0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:23.045704 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:23.045655    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 18:59:23.054628 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:23.045946    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 18:59:23.055106 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:23.045996    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:23.058624 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:23.046021    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:23.501720 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:23.488501    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:23.521555 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:23.521501    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:23.643026 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:23.642997    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:59:23.671364 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:23.671241    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:59:23.954709 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:23.847827    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:59:24.167621 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:24.148820    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:24.168106 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:24.155844    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:24.208375 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:24.208291    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:59:24.222430 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:24.221880    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:59:24.240078 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:24.240033    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:59:27.283159 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.283124    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"]
Oct 30 18:59:27.287440 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.287415    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.325360 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.325326    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-dshm\") pod \"distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") " pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.325520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.325377    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-tmp-dir\") pod \"distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") " pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.325520 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.325404    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-home\") pod \"distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") " pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.325659 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.325533    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7rsv\" (UniqueName: \"kubernetes.io/projected/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-kube-api-access-r7rsv\") pod \"distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") " pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.325778 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.325756    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-tls-certs\") pod \"distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") " pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.325840 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.325795    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-model-cache\") pod \"distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") " pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.364561 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.364527    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"]
Oct 30 18:59:27.385487 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.385461    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-3b6ad6\"/\"distinct-2-w5-3b6ad6-kserve-self-signed-certs\""
Oct 30 18:59:27.426877 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.426850    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-tls-certs\") pod \"distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") " pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.427038 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.426885    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-model-cache\") pod \"distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") " pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.427038 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.426957    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-dshm\") pod \"distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") " pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.427038 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.426996    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-tmp-dir\") pod \"distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") " pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.427038 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.427025    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-home\") pod \"distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") " pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.427232 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.427072    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7rsv\" (UniqueName: \"kubernetes.io/projected/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-kube-api-access-r7rsv\") pod \"distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") " pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.427503 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.427470    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-home\") pod \"distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") " pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.427656 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.427508    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-tmp-dir\") pod \"distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") " pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.427741 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.427654    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-model-cache\") pod \"distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") " pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.429521 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.429480    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-tls-certs\") pod \"distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") " pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.429652 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.429523    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-dshm\") pod \"distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") " pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.481968 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.481932    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7rsv\" (UniqueName: \"kubernetes.io/projected/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-kube-api-access-r7rsv\") pod \"distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") " pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.601733 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.601648    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:27.974112 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:59:27.974080    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17c5e93f_ffe3_47c7_9c29_b2bf0e6e9a06.slice/crio-894642a5e73df072b906a1ec08ef88bb887dd831bd0240dae61ba0c5434e6c01 WatchSource:0}: Error finding container 894642a5e73df072b906a1ec08ef88bb887dd831bd0240dae61ba0c5434e6c01: Status 404 returned error can't find the container with id 894642a5e73df072b906a1ec08ef88bb887dd831bd0240dae61ba0c5434e6c01
Oct 30 18:59:27.974513 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:27.974487    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"]
Oct 30 18:59:28.683243 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:28.683201    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" event={"ID":"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06","Type":"ContainerStarted","Data":"6e2eaa7ecc2884896fe4d5a66a2f65b01874862ec11e1bd876185344921f2f1e"}
Oct 30 18:59:28.684303 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:28.684262    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" event={"ID":"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06","Type":"ContainerStarted","Data":"894642a5e73df072b906a1ec08ef88bb887dd831bd0240dae61ba0c5434e6c01"}
Oct 30 18:59:28.796482 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:28.796416    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podStartSLOduration=1.7963998700000001 podStartE2EDuration="1.79639987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:28.796123534 +0000 UTC m=+1366.321998630" watchObservedRunningTime="2026-10-30 18:59:28.79639987 +0000 UTC m=+1366.322274967"
Oct 30 18:59:29.601981 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:29.601948    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:35.531589 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:59:35.531549    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 18:59:37.601782 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:37.601744    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:37.616530 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:37.616501    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:37.748197 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:37.748162    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:40.279092 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.279055    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"]
Oct 30 18:59:40.284393 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.284370    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.323877 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.323848    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"]
Oct 30 18:59:40.327808 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.327785    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.377755 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.377722    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"]
Oct 30 18:59:40.382939 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.382903    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"]
Oct 30 18:59:40.421052 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.421018    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-workload-certs\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.421227 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.421062    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-istio-data\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.421227 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.421090    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9klsg\" (UniqueName: \"kubernetes.io/projected/0fe6b941-e299-4145-b798-3d590e53048e-kube-api-access-9klsg\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.421307 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.421230    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-credential-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.421307 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.421269    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-workload-certs\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.421374 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.421309    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe6b941-e299-4145-b798-3d590e53048e-istio-podinfo\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.421374 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.421347    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-envoy\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.421446 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.421388    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-token\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.421446 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.421424    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7tj6\" (UniqueName: \"kubernetes.io/projected/90f8bdc0-d655-4c71-a348-9fa661ece9c6-kube-api-access-m7tj6\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.421509 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.421464    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-workload-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.421509 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.421495    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe6b941-e299-4145-b798-3d590e53048e-istio-token\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.421636 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.421564    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-podinfo\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.421749 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.421710    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe6b941-e299-4145-b798-3d590e53048e-istiod-ca-cert\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.422123 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.422080    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-istio-envoy\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.422241 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.422220    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-credential-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.422297 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.422254    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-data\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.422346 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.422320    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-workload-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.422404 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.422345    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istiod-ca-cert\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.523414 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.523388    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe6b941-e299-4145-b798-3d590e53048e-istiod-ca-cert\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.523592 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.523434    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-istio-envoy\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.523592 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.523486    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-credential-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.523592 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.523503    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-data\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.523592 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.523565    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-workload-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.523592 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.523590    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istiod-ca-cert\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.523836 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.523652    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-workload-certs\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.523836 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.523749    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-istio-data\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.523836 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.523792    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9klsg\" (UniqueName: \"kubernetes.io/projected/0fe6b941-e299-4145-b798-3d590e53048e-kube-api-access-9klsg\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.523941 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.523854    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-credential-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.523941 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.523878    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-workload-certs\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.523941 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.523915    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe6b941-e299-4145-b798-3d590e53048e-istio-podinfo\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.524053 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.523952    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-envoy\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.524053 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.523994    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-token\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.524131 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.524051    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7tj6\" (UniqueName: \"kubernetes.io/projected/90f8bdc0-d655-4c71-a348-9fa661ece9c6-kube-api-access-m7tj6\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.524131 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.524091    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-workload-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.524131 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.524121    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe6b941-e299-4145-b798-3d590e53048e-istio-token\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.524256 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.524191    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-podinfo\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.524256 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.524222    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe6b941-e299-4145-b798-3d590e53048e-istiod-ca-cert\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.524354 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.524337    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-credential-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.524779 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.524756    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-credential-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.524936 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.524810    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-data\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.525037 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.524945    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-istio-data\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.525093 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.525038    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-workload-certs\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.525328 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.525296    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-workload-certs\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.525328 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.525312    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-workload-socket\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.525469 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.525308    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-workload-socket\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.525746 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.525726    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istiod-ca-cert\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.526663 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.526640    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-istio-envoy\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.527131 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.527109    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe6b941-e299-4145-b798-3d590e53048e-istio-podinfo\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.527326 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.527308    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-envoy\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.527529 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.527512    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-podinfo\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.580302 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.580227    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-token\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.580648 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.580625    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7tj6\" (UniqueName: \"kubernetes.io/projected/90f8bdc0-d655-4c71-a348-9fa661ece9c6-kube-api-access-m7tj6\") pod \"e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") " pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:40.580847 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.580825    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9klsg\" (UniqueName: \"kubernetes.io/projected/0fe6b941-e299-4145-b798-3d590e53048e-kube-api-access-9klsg\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.581333 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.581310    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe6b941-e299-4145-b798-3d590e53048e-istio-token\") pod \"e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") " pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.600714 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.600689    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:40.669882 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:40.669849    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:41.029563 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:41.029530    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"]
Oct 30 18:59:41.030688 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:59:41.030661    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fe6b941_e299_4145_b798_3d590e53048e.slice/crio-09256580230e3dd00c4932bdae719ab94e15980a492834cff9082a8c88f2a2d0 WatchSource:0}: Error finding container 09256580230e3dd00c4932bdae719ab94e15980a492834cff9082a8c88f2a2d0: Status 404 returned error can't find the container with id 09256580230e3dd00c4932bdae719ab94e15980a492834cff9082a8c88f2a2d0
Oct 30 18:59:41.033162 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:41.033128    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:59:41.033228 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:41.033194    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:59:41.033268 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:41.033244    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:59:41.087038 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:41.087010    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"]
Oct 30 18:59:41.089818 ip-10-0-140-122 kubenswrapper[4544]: W1030 18:59:41.089793    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90f8bdc0_d655_4c71_a348_9fa661ece9c6.slice/crio-ccf91ba3f0e3755641a54a6ee8176622cd2442b15e90fd3b47dc1d7dd94f7c35 WatchSource:0}: Error finding container ccf91ba3f0e3755641a54a6ee8176622cd2442b15e90fd3b47dc1d7dd94f7c35: Status 404 returned error can't find the container with id ccf91ba3f0e3755641a54a6ee8176622cd2442b15e90fd3b47dc1d7dd94f7c35
Oct 30 18:59:41.092644 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:41.092594    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:59:41.092754 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:41.092698    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:59:41.092754 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:41.092743    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 18:59:41.787651 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:41.787006    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq" event={"ID":"0fe6b941-e299-4145-b798-3d590e53048e","Type":"ContainerStarted","Data":"f596c62e9b83ec5311a9e443db8e9b6ffefc2026d0ff82c2f21ef99c65a21616"}
Oct 30 18:59:41.787651 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:41.787075    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq" event={"ID":"0fe6b941-e299-4145-b798-3d590e53048e","Type":"ContainerStarted","Data":"09256580230e3dd00c4932bdae719ab94e15980a492834cff9082a8c88f2a2d0"}
Oct 30 18:59:41.790017 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:41.789988    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g" event={"ID":"90f8bdc0-d655-4c71-a348-9fa661ece9c6","Type":"ContainerStarted","Data":"f739ceadc5a512377ca2b7119646e928233e7244717e5e3eb6d71b8588bf7864"}
Oct 30 18:59:41.790155 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:41.790049    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g" event={"ID":"90f8bdc0-d655-4c71-a348-9fa661ece9c6","Type":"ContainerStarted","Data":"ccf91ba3f0e3755641a54a6ee8176622cd2442b15e90fd3b47dc1d7dd94f7c35"}
Oct 30 18:59:41.901032 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:41.900962    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq" podStartSLOduration=1.9009428659999998 podStartE2EDuration="1.900942866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:41.899295075 +0000 UTC m=+1379.425170173" watchObservedRunningTime="2026-10-30 18:59:41.900942866 +0000 UTC m=+1379.426817963"
Oct 30 18:59:42.021634 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:42.021543    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g" podStartSLOduration=2.021520589 podStartE2EDuration="2.021520589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:42.012366693 +0000 UTC m=+1379.538241784" watchObservedRunningTime="2026-10-30 18:59:42.021520589 +0000 UTC m=+1379.547395693"
Oct 30 18:59:42.606627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:42.606457    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 18:59:42.606627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:42.606518    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq" podUID="0fe6b941-e299-4145-b798-3d590e53048e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 18:59:42.625162 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:42.623223    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:42.680093 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:42.679969    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 18:59:42.680093 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:42.680028    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g" podUID="90f8bdc0-d655-4c71-a348-9fa661ece9c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 18:59:42.680380 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:42.678037    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:43.611628 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:43.611049    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 18:59:43.611628 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:43.611108    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq" podUID="0fe6b941-e299-4145-b798-3d590e53048e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 18:59:43.677627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:43.676703    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 18:59:43.677627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:43.676780    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g" podUID="90f8bdc0-d655-4c71-a348-9fa661ece9c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 18:59:44.606064 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:44.606020    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 18:59:44.606314 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:44.606285    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq" podUID="0fe6b941-e299-4145-b798-3d590e53048e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 18:59:44.674179 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:44.674141    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 18:59:44.674815 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:44.674767    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g" podUID="90f8bdc0-d655-4c71-a348-9fa661ece9c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 18:59:45.612628 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:45.608198    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 18:59:45.612628 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:45.608261    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq" podUID="0fe6b941-e299-4145-b798-3d590e53048e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 18:59:45.685628 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:45.684853    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 18:59:45.685628 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:45.684919    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g" podUID="90f8bdc0-d655-4c71-a348-9fa661ece9c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 18:59:46.620685 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:46.608816    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 18:59:46.620685 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:46.608876    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq" podUID="0fe6b941-e299-4145-b798-3d590e53048e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 18:59:46.680624 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:46.679713    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 18:59:46.680624 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:46.679775    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g" podUID="90f8bdc0-d655-4c71-a348-9fa661ece9c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 18:59:47.602260 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:47.602159    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 18:59:47.602260 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:47.602226    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq" podUID="0fe6b941-e299-4145-b798-3d590e53048e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 18:59:47.672145 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:47.672104    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 18:59:47.672343 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:47.672169    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g" podUID="90f8bdc0-d655-4c71-a348-9fa661ece9c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 18:59:48.204876 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:48.204821    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"]
Oct 30 18:59:48.205107 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:48.205072    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck" podUID="bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" containerName="istio-proxy" containerID="cri-o://dec02520188a5d332a315216dbb185b0c4ebca740bc95bc38ed45c6cebd15e79" gracePeriod=30
Oct 30 18:59:49.631629 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:49.630827    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:49.652350 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:49.652310    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:59:49.652628 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:49.652585    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq" podUID="0fe6b941-e299-4145-b798-3d590e53048e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:49.760472 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:49.760420    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:59:49.760867 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:49.760830    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g" podUID="90f8bdc0-d655-4c71-a348-9fa661ece9c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:49.787841 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:49.787778    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:50.095638 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:50.094836    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:50.096174 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:50.096137    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": context deadline exceeded"
Oct 30 18:59:50.099630 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:50.098762    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:50.099630 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:50.099059    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 18:59:50.104632 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:50.099941    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 18:59:50.580756 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:50.580697    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:50.697509 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:50.696705    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:59:50.697509 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:50.696779    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq" podUID="0fe6b941-e299-4145-b798-3d590e53048e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:50.850683 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:50.847835    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:50.850683 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:50.848442    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:50.850683 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:50.850285    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:59:50.850683 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:50.850328    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g" podUID="90f8bdc0-d655-4c71-a348-9fa661ece9c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:51.121687 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.105367    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 18:59:51.125626 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.111702    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 18:59:51.178676 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.129748    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:59:51.178676 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.174814    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:59:51.178676 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.174920    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:51.182682 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.141690    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:51.208629 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.141743    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": context deadline exceeded"
Oct 30 18:59:51.237634 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.191062    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:51.245627 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.241736    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:59:51.258760 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.258726    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:59:51.305645 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.141768    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 18:59:51.468914 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.420070    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 18:59:51.513802 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.513756    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:51.528629 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.518070    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:59:51.625638 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.589879    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 18:59:51.625638 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.590376    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:59:51.733794 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.733427    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:51.734318 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.733904    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:59:51.970626 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.848127    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:51.989744 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.869712    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:59:51.989744 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:51.982258    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g" podUID="90f8bdc0-d655-4c71-a348-9fa661ece9c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:52.157462 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:52.152222    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 18:59:52.157887 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:52.157854    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:52.188842 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:52.184077    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:52.225633 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:52.217265    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": context deadline exceeded"
Oct 30 18:59:52.429789 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:52.429750    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:59:52.430116 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:52.430073    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk" podUID="0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:52.462156 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:52.462097    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 18:59:52.638846 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:52.638785    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 18:59:52.740682 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:52.739833    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:53.140629 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:53.140258    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 18:59:53.206630 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:53.173871    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:59:53.468739 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:53.404728    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g" podUID="90f8bdc0-d655-4c71-a348-9fa661ece9c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:53.468739 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:53.208578    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 18:59:53.468739 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:53.371415    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:53.468739 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:53.207244    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:53.512313 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:53.512251    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:53.552654 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:53.552173    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:53.650732 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:53.644594    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:53.745494 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:53.745383    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 18:59:54.347626 ip-10-0-140-122 kubenswrapper[4544]: E1030 18:59:54.344559    4544 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.748s"
Oct 30 18:59:54.382640 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:54.377222    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:59:54.677010 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:54.676911    4544 generic.go:356] "Generic (PLEG): container finished" podID="bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" containerID="dec02520188a5d332a315216dbb185b0c4ebca740bc95bc38ed45c6cebd15e79" exitCode=0
Oct 30 18:59:54.951684 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:54.900458    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck" event={"ID":"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41","Type":"ContainerDied","Data":"dec02520188a5d332a315216dbb185b0c4ebca740bc95bc38ed45c6cebd15e79"}
Oct 30 18:59:55.369639 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:55.362109    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 18:59:55.369639 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:55.367159    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 18:59:55.369639 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:55.367735    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 18:59:55.409316 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:55.409262    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:55.410035 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:55.410002    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:55.410529 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:55.410504    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": context deadline exceeded"
Oct 30 18:59:55.410924 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:55.410902    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 18:59:55.411287 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:55.411258    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": context deadline exceeded"
Oct 30 18:59:55.415710 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:55.415685    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 18:59:55.457160 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:55.457121    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:59:55.457430 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:55.457404    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g" podUID="90f8bdc0-d655-4c71-a348-9fa661ece9c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:55.599976 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:55.546942    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/health\": context deadline exceeded"
Oct 30 18:59:55.673780 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:55.673668    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:56.024096 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.011902    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 18:59:56.205793 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.205744    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 18:59:56.320918 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.320823    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/health\": context deadline exceeded"
Oct 30 18:59:56.353729 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.353688    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:59:56.355703 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.355681    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 18:59:56.366450 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.365774    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:56.375929 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.374969    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:56.415654 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.412841    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:56.415654 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.413537    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:56.421405 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.419030    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:56.431061 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.431028    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:59:56.447705 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.447676    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-credential-socket\") pod \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") "
Oct 30 18:59:56.448817 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.448000    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-workload-socket\") pod \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") "
Oct 30 18:59:56.449029 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.449009    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-token\" (UniqueName: \"kubernetes.io/projected/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-token\") pod \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") "
Oct 30 18:59:56.449246 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.449230    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-envoy\") pod \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") "
Oct 30 18:59:56.449361 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.449348    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-workload-certs\") pod \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") "
Oct 30 18:59:56.452744 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.452721    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-podinfo\") pod \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") "
Oct 30 18:59:56.479811 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.478076    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-kube-api-access-8mdnr\" (UniqueName: \"kubernetes.io/projected/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-kube-api-access-8mdnr\") pod \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") "
Oct 30 18:59:56.479811 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.478284    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istiod-ca-cert\") pod \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") "
Oct 30 18:59:56.479811 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.478702    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-data\") pod \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\" (UID: \"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41\") "
Oct 30 18:59:56.491637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.458090    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-credential-socket" pod "bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" (UID: "bc5bccb4-412b-4142-8cf5-8e4f99c6ef41"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:59:56.491637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.458304    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-workload-socket" pod "bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" (UID: "bc5bccb4-412b-4142-8cf5-8e4f99c6ef41"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:59:56.491637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.471671    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-token" pod "bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" (UID: "bc5bccb4-412b-4142-8cf5-8e4f99c6ef41"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:56.491637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.480789    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-workload-certs" pod "bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" (UID: "bc5bccb4-412b-4142-8cf5-8e4f99c6ef41"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:59:56.491637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.484073    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-podinfo" pod "bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" (UID: "bc5bccb4-412b-4142-8cf5-8e4f99c6ef41"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:59:56.491637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.471188    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 18:59:56.491637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.485259    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 18:59:56.491637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.486182    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istiod-ca-cert" pod "bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" (UID: "bc5bccb4-412b-4142-8cf5-8e4f99c6ef41"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:56.491637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.487462    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:56.491637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.487488    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:56.491637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.487504    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:56.491637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.487519    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:56.491637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.487548    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:56.491637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.487563    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:56.491637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.485542    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 18:59:56.491637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.489424    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 18:59:56.491637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.489872    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-data" pod "bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" (UID: "bc5bccb4-412b-4142-8cf5-8e4f99c6ef41"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:59:56.491637 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.489940    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-envoy" pod "bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" (UID: "bc5bccb4-412b-4142-8cf5-8e4f99c6ef41"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:59:56.505121 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.505087    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:59:56.505249 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.505144    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g" podUID="90f8bdc0-d655-4c71-a348-9fa661ece9c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:56.556273 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.556235    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-kube-api-access-8mdnr" pod "bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" (UID: "bc5bccb4-412b-4142-8cf5-8e4f99c6ef41"). InnerVolumeSpecName "kube-api-access-8mdnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:56.581976 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.581885    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 18:59:56.590645 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.590566    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:56.590645 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.590622    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:56.590645 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.590642    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mdnr\" (UniqueName: \"kubernetes.io/projected/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41-kube-api-access-8mdnr\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:56.705413 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:56.705353    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:57.029218 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:57.029181    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"
Oct 30 18:59:57.029763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:57.029273    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck" event={"ID":"bc5bccb4-412b-4142-8cf5-8e4f99c6ef41","Type":"ContainerDied","Data":"96cd700c050539254af8d3c95e4f6a13d9c7f2777152b85c62ed613e50e70ac7"}
Oct 30 18:59:57.029763 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:57.029593    4544 scope.go:122] "RemoveContainer" containerID="dec02520188a5d332a315216dbb185b0c4ebca740bc95bc38ed45c6cebd15e79"
Oct 30 18:59:57.030804 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:57.030779    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 18:59:57.321330 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:57.321221    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"]
Oct 30 18:59:57.329487 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:57.329458    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-c5dck"]
Oct 30 18:59:58.604080 ip-10-0-140-122 kubenswrapper[4544]: I1030 18:59:58.604041    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" path="/var/lib/kubelet/pods/bc5bccb4-412b-4142-8cf5-8e4f99c6ef41/volumes"
Oct 30 19:00:05.532635 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:00:05.532567    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:00:10.271419 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.271382    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"]
Oct 30 19:00:10.272000 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.271984    4544 cpu_manager.go:414] "containerMap: removing container" podUID="bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" containerName="istio-proxy"
Oct 30 19:00:10.272044 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.272000    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" containerName="istio-proxy"
Oct 30 19:00:10.272127 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.272118    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc5bccb4-412b-4142-8cf5-8e4f99c6ef41" containerName="istio-proxy"
Oct 30 19:00:10.275614 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.275582    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.318036 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.318007    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"]
Oct 30 19:00:10.322146 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.322127    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"]
Oct 30 19:00:10.322275 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.322252    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.325907 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.325889    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"]
Oct 30 19:00:10.326025 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.326009    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.330804 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.330787    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.371558 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.371518    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"]
Oct 30 19:00:10.375579 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.375546    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-credential-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.375579 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.375592    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-istio-envoy\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.375839 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.375671    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71bee142-6883-471f-933d-18f2424a1bac-istio-podinfo\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.375839 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.375724    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71bee142-6883-471f-933d-18f2424a1bac-istiod-ca-cert\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.375839 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.375749    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p255p\" (UniqueName: \"kubernetes.io/projected/71bee142-6883-471f-933d-18f2424a1bac-kube-api-access-p255p\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.376012 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.375902    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-istio-data\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.376012 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.375942    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71bee142-6883-471f-933d-18f2424a1bac-istio-token\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.376123 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.376038    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-workload-certs\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.376123 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.376115    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-workload-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.376584 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.376560    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"]
Oct 30 19:00:10.425922 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.425889    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"]
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.532062    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-istio-envoy\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.532131    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22256\" (UniqueName: \"kubernetes.io/projected/94fb0221-0a90-421d-9504-80067ac621ca-kube-api-access-22256\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.532245    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-workload-socket\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.532278    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71bee142-6883-471f-933d-18f2424a1bac-istiod-ca-cert\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.532341    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-workload-certs\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.532405    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-workload-socket\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.532433    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94fb0221-0a90-421d-9504-80067ac621ca-istio-podinfo\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.532484    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-workload-certs\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.532556    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-credential-socket\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.532624    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-data\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.532661    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-workload-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.532703    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71bee142-6883-471f-933d-18f2424a1bac-istio-podinfo\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.532734    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-credential-socket\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.532797    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istiod-ca-cert\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.532882    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-workload-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.532908    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-token\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.532979    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-credential-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.533031    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-credential-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.533064    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p255p\" (UniqueName: \"kubernetes.io/projected/71bee142-6883-471f-933d-18f2424a1bac-kube-api-access-p255p\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.533118    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-podinfo\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.533152    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zskxs\" (UniqueName: \"kubernetes.io/projected/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-kube-api-access-zskxs\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.533222    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-envoy\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.533284    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-envoy\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.533333    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-data\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.533379    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94fb0221-0a90-421d-9504-80067ac621ca-istiod-ca-cert\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.533450    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-istio-data\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.533502    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-podinfo\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.533587    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istiod-ca-cert\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.534203    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-istio-data\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.535168    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71bee142-6883-471f-933d-18f2424a1bac-istiod-ca-cert\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.535568    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-workload-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.535742    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-istio-envoy\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.535865    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jggwn\" (UniqueName: \"kubernetes.io/projected/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-kube-api-access-jggwn\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.535933    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71bee142-6883-471f-933d-18f2424a1bac-istio-token\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.535977    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-istio-data\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.536053    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-workload-certs\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.536094    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-token\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.536132    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-workload-certs\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.536165    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94fb0221-0a90-421d-9504-80067ac621ca-istio-token\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.536990    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-credential-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.537495    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-istio-envoy\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.538721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.537790    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-workload-certs\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.544024 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.543987    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"]
Oct 30 19:00:10.561321 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.561290    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71bee142-6883-471f-933d-18f2424a1bac-istio-podinfo\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.623731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.623634    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p255p\" (UniqueName: \"kubernetes.io/projected/71bee142-6883-471f-933d-18f2424a1bac-kube-api-access-p255p\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.627484 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.627165    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71bee142-6883-471f-933d-18f2424a1bac-istio-token\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639089    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-envoy\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639154    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-envoy\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639184    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-data\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639225    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94fb0221-0a90-421d-9504-80067ac621ca-istiod-ca-cert\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639268    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-podinfo\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639319    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istiod-ca-cert\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639342    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-istio-envoy\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639372    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jggwn\" (UniqueName: \"kubernetes.io/projected/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-kube-api-access-jggwn\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639423    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-istio-data\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639481    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-workload-certs\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639510    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-token\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639564    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94fb0221-0a90-421d-9504-80067ac621ca-istio-token\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639662    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22256\" (UniqueName: \"kubernetes.io/projected/94fb0221-0a90-421d-9504-80067ac621ca-kube-api-access-22256\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639738    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-workload-socket\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639775    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-workload-certs\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639821    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-workload-socket\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639845    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94fb0221-0a90-421d-9504-80067ac621ca-istio-podinfo\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639878    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-workload-certs\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639946    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-credential-socket\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.639987    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-data\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.640025    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-credential-socket\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.640067    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istiod-ca-cert\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.640144    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-workload-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.640169    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-token\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.640233    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-credential-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.640306    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-podinfo\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.641169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.640345    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zskxs\" (UniqueName: \"kubernetes.io/projected/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-kube-api-access-zskxs\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.644915 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.644282    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-credential-socket\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.644915 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.644529    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-workload-socket\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.645574 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.645550    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-workload-certs\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.646559 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.646536    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istiod-ca-cert\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.650219 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.646963    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-workload-certs\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.650404 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.648935    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-credential-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.651828 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.651800    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-workload-socket\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.652441 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.652419    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-credential-socket\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.652896 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.652868    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-data\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.653115 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.653096    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-istio-data\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.654986 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.654962    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-data\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.655360 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.655330    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istiod-ca-cert\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.656019 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.655991    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94fb0221-0a90-421d-9504-80067ac621ca-istiod-ca-cert\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.656779 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.656756    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-workload-certs\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.657484 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.657452    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-workload-socket\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.661994 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.661972    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-podinfo\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.662165 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.662146    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-istio-envoy\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.670770 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.670321    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94fb0221-0a90-421d-9504-80067ac621ca-istio-podinfo\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.671632 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.670889    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-envoy\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.671632 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.671260    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-envoy\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.673132 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.673087    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-podinfo\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.753638 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.753384    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zskxs\" (UniqueName: \"kubernetes.io/projected/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-kube-api-access-zskxs\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.784831 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.784778    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jggwn\" (UniqueName: \"kubernetes.io/projected/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-kube-api-access-jggwn\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.786137 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.786097    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-token\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:10.788189 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.788127    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22256\" (UniqueName: \"kubernetes.io/projected/94fb0221-0a90-421d-9504-80067ac621ca-kube-api-access-22256\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.803212 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.803140    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-token\") pod \"e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") " pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.821234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.821150    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94fb0221-0a90-421d-9504-80067ac621ca-istio-token\") pod \"e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") " pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.889463 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.889003    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:10.936993 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.936751    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:10.945359 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.945167    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:10.953672 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:10.953463    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:11.540634 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:11.538199    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:00:11.540634 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:11.538299    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:00:11.540634 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:11.538342    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:00:11.615197 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:11.615164    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"]
Oct 30 19:00:12.236808 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:12.235472    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:00:12.236808 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:12.235594    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:00:12.236808 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:12.235657    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:00:12.338497 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:12.332924    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:00:12.338497 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:12.333016    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:00:12.338497 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:12.333064    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:00:12.391140 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:12.387644    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l" event={"ID":"76337bdf-89a0-4e5d-9cb0-437962f9dc8c","Type":"ContainerStarted","Data":"6f8f38e83924b4f22fa97adce2fc9eff0a56966b51f9542d2dda452488ce3747"}
Oct 30 19:00:12.391140 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:00:12.387787    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47a6ccf3_4ee4_41f4_aeeb_35e0c996aecd.slice/crio-d669ec4cc694ee00970dea1000e63de9ecda3ee871db42211cacfa86ab2c4475 WatchSource:0}: Error finding container d669ec4cc694ee00970dea1000e63de9ecda3ee871db42211cacfa86ab2c4475: Status 404 returned error can't find the container with id d669ec4cc694ee00970dea1000e63de9ecda3ee871db42211cacfa86ab2c4475
Oct 30 19:00:12.412639 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:12.412021    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:12.449970 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:12.449115    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:12.516895 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:12.515444    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv" event={"ID":"71bee142-6883-471f-933d-18f2424a1bac","Type":"ContainerStarted","Data":"d1c16ac4c8fb89399f1256b20fe8938e1924321ad073d2e008986a26d58c3ecd"}
Oct 30 19:00:12.543855 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:12.543800    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"]
Oct 30 19:00:12.545847 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:12.545803    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:00:12.545953 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:12.545885    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:00:12.545953 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:12.545936    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:00:12.879157 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:12.825083    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv" podStartSLOduration=2.825058851 podStartE2EDuration="2.825058851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:12.655453625 +0000 UTC m=+1410.181328727" watchObservedRunningTime="2026-10-30 19:00:12.825058851 +0000 UTC m=+1410.350933950"
Oct 30 19:00:12.984825 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:12.975058    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"]
Oct 30 19:00:12.984825 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:12.975131    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:12.984825 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:12.975160    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"]
Oct 30 19:00:13.171666 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:13.154633    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:13.203822 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:13.201992    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 19:00:13.396508 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:13.393831    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 19:00:13.445179 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:13.412864    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:13.445179 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:13.413219    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 19:00:13.530122 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:13.529708    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": context deadline exceeded"
Oct 30 19:00:13.530122 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:13.529804    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 19:00:13.531649 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:13.530406    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:13.531649 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:13.530756    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 19:00:13.577097 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:13.577059    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:13.893797 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:13.893705    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 19:00:13.959870 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:13.959825    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:00:13.960104 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:13.960076    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv" podUID="71bee142-6883-471f-933d-18f2424a1bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:00:14.150686 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:14.139478    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv" event={"ID":"71bee142-6883-471f-933d-18f2424a1bac","Type":"ContainerStarted","Data":"edbf6c2e5bb0b8e6ea666133fc8cc68d9852bdb2ae1aacb41f93293cdca3c13b"}
Oct 30 19:00:14.213625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:14.213283    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:14.247750 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:14.230034    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 19:00:14.422523 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:14.394178    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:14.422523 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:14.395471    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 19:00:14.422523 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:14.418915    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:14.689716 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:14.638747    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:14.818872 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:14.818836    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 19:00:14.892655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:14.845790    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:14.892655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:14.845989    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:14.923333 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:14.910561    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:14.943253 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:14.926521    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:00:14.943253 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:14.926590    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv" podUID="71bee142-6883-471f-933d-18f2424a1bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:00:15.168907 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:15.162908    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:15.244228 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:15.244134    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 19:00:15.418629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:15.417908    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:15.436746 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:15.436701    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:00:15.470334 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:15.470280    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:15.611636 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:15.608506    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 19:00:15.641740 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:15.641696    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:15.728553 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:15.707248    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8" event={"ID":"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd","Type":"ContainerStarted","Data":"d669ec4cc694ee00970dea1000e63de9ecda3ee871db42211cacfa86ab2c4475"}
Oct 30 19:00:15.728553 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:15.707360    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 19:00:15.728553 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:15.707400    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks" event={"ID":"94fb0221-0a90-421d-9504-80067ac621ca","Type":"ContainerStarted","Data":"c71d65a7a9913b8c028f8cd22460207da43735ac8f240688df8f8fee2f13586c"}
Oct 30 19:00:15.804652 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:15.787093    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l" event={"ID":"76337bdf-89a0-4e5d-9cb0-437962f9dc8c","Type":"ContainerStarted","Data":"6e401091e3a31d2b6ac2a5ce0d72426b2ee7e9d849c3a991dbcd5dff87300818"}
Oct 30 19:00:15.853648 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:15.847689    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:15.944718 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:15.941998    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:00:15.944718 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:15.942070    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv" podUID="71bee142-6883-471f-933d-18f2424a1bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:00:16.002630 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:15.981365    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:00:16.031628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:16.021946    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks" podUID="94fb0221-0a90-421d-9504-80067ac621ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:00:16.031628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:16.016650    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 19:00:16.031628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:16.016589    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:16.057625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:16.050279    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:16.142658 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:15.997596    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks" podStartSLOduration=5.997574441 podStartE2EDuration="5.997574441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:15.559738166 +0000 UTC m=+1413.085613263" watchObservedRunningTime="2026-10-30 19:00:15.997574441 +0000 UTC m=+1413.523449538"
Oct 30 19:00:16.142658 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:16.114925    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:00:16.142658 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:16.114977    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l" podUID="76337bdf-89a0-4e5d-9cb0-437962f9dc8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:00:16.184625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:16.183199    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:16.184625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:16.184482    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 19:00:16.282683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:16.276255    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:16.282683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:16.276355    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 19:00:16.509571 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:16.487891    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:16.509571 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:16.489109    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:16.649751 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:16.630305    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l" podStartSLOduration=6.630279193 podStartE2EDuration="6.630279193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:16.114725875 +0000 UTC m=+1413.640600972" watchObservedRunningTime="2026-10-30 19:00:16.630279193 +0000 UTC m=+1414.156154294"
Oct 30 19:00:16.687284 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:16.650692    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:16.855630 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:16.853079    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:17.004684 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.004180    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:00:17.004684 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.004249    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l" podUID="76337bdf-89a0-4e5d-9cb0-437962f9dc8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:00:17.004684 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.004448    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:00:17.004684 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.004473    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv" podUID="71bee142-6883-471f-933d-18f2424a1bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:00:17.004684 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.004531    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:00:17.004684 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.004555    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks" podUID="94fb0221-0a90-421d-9504-80067ac621ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:00:17.074683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.036444    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:17.132301 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.132152    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:00:17.132301 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.132217    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75" podUID="be77624e-eeb7-44f4-8d9e-dfd58897e555" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:17.207628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.197997    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 19:00:17.232738 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.232649    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8" event={"ID":"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd","Type":"ContainerStarted","Data":"6e83192df4103c4acb4a3899f2a4fed65fad77013d35a74c4e7d440c31727ed7"}
Oct 30 19:00:17.292065 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.290416    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:17.292616 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.292566    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/health\": context deadline exceeded"
Oct 30 19:00:17.330629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.305231    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks" event={"ID":"94fb0221-0a90-421d-9504-80067ac621ca","Type":"ContainerStarted","Data":"10fcff2542e940f4afbf65b26b2e7d2a7df9cda0ca4e04b68c3e6243b8eca32c"}
Oct 30 19:00:17.489627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.489557    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:17.502999 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.497064    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:17.632683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.632536    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:17.674278 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.674207    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8" podStartSLOduration=7.674181793 podStartE2EDuration="7.674181793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:17.309129952 +0000 UTC m=+1414.835005049" watchObservedRunningTime="2026-10-30 19:00:17.674181793 +0000 UTC m=+1415.200056892"
Oct 30 19:00:17.702369 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.701785    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:17.912686 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.887191    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:18.033667 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.938906    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:18.033667 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.959522    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:00:18.033667 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.021884    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8" podUID="47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:00:18.033667 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:17.959574    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:00:18.033667 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.021949    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv" podUID="71bee142-6883-471f-933d-18f2424a1bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:00:18.033667 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.021670    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:00:18.033667 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.021990    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks" podUID="94fb0221-0a90-421d-9504-80067ac621ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:00:18.033667 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.021714    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:00:18.033667 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.022028    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l" podUID="76337bdf-89a0-4e5d-9cb0-437962f9dc8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:00:18.067628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.044798    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:18.067628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.053907    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 19:00:18.213242 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.213142    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:18.331362 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.331318    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 19:00:18.428089 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.428057    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 19:00:18.546626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.535667    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:18.667628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.661744    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 19:00:18.850689 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.814555    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:18.910633 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.904617    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": context deadline exceeded"
Oct 30 19:00:19.128216 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.937565    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:00:19.130319 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:19.129482    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv" podUID="71bee142-6883-471f-933d-18f2424a1bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:00:19.130522 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:19.117797    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:19.130982 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:19.005949    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:19.131128 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.947743    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:00:19.131281 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:19.131231    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks" podUID="94fb0221-0a90-421d-9504-80067ac621ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:00:19.131421 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.947795    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:00:19.131531 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:19.131507    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8" podUID="47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:00:19.131671 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:18.955234    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:00:19.131825 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:19.131786    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l" podUID="76337bdf-89a0-4e5d-9cb0-437962f9dc8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:00:19.232524 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:19.232483    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:19.339853 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:19.339809    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:19.454857 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:19.454710    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:00:19.574953 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:19.557701    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 19:00:19.682931 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:19.682871    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:19.722764 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:00:19.722641    4544 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.118s"
Oct 30 19:00:19.723158 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:19.723114    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 19:00:19.919721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:19.912703    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:19.966003 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:19.960795    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:00:19.966003 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:19.960862    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8" podUID="47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:00:19.966003 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:19.960941    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:00:19.966003 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:19.960964    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv" podUID="71bee142-6883-471f-933d-18f2424a1bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:00:20.025251 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.006703    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:00:20.025251 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.006765    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks" podUID="94fb0221-0a90-421d-9504-80067ac621ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:00:20.025251 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.022240    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:00:20.025251 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.022343    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l" podUID="76337bdf-89a0-4e5d-9cb0-437962f9dc8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:00:20.147625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.146714    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:20.286706 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.268514    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:20.382626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.381771    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:20.433736 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.433687    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:20.577708 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.574695    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:20.577708 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.575151    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 19:00:20.585640 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.584836    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 19:00:20.711133 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.684517    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:20.942848 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.932562    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:00:20.942848 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.932649    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv" podUID="71bee142-6883-471f-933d-18f2424a1bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:00:20.942848 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.934258    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": context deadline exceeded"
Oct 30 19:00:20.968137 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.968095    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:00:20.968370 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.968343    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8" podUID="47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:00:20.975691 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.968685    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:00:20.997855 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.997810    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l" podUID="76337bdf-89a0-4e5d-9cb0-437962f9dc8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:00:21.001624 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:20.968728    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:00:21.009627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:21.005675    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks" podUID="94fb0221-0a90-421d-9504-80067ac621ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:00:21.163627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:21.161292    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 19:00:21.292705 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:21.283210    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:21.486630 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:21.457053    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 19:00:21.623425 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:21.623292    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 19:00:21.624664 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:21.499403    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 19:00:21.625297 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:21.502813    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:21.625492 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:21.594545    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:21.650467 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:21.595212    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:21.721439 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:21.721393    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 19:00:21.981692 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:21.953704    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:00:21.981692 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:21.953765    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8" podUID="47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:00:21.982025 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:21.981989    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:00:22.012790 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.012751    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv" podUID="71bee142-6883-471f-933d-18f2424a1bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:00:22.055724 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:21.988152    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:22.074625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.007764    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:00:22.074625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.055966    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l" podUID="76337bdf-89a0-4e5d-9cb0-437962f9dc8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:00:22.200635 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.178728    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:22.227867 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.227830    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:00:22.228130 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.228100    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks" podUID="94fb0221-0a90-421d-9504-80067ac621ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:00:22.294660 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.287763    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:22.657688 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.623479    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 19:00:22.657688 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.630718    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:22.657688 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.650629    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:22.657688 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.651778    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:00:22.736633 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.730578    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:22.907625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.893686    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:00:22.907625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.893770    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv" podUID="71bee142-6883-471f-933d-18f2424a1bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:00:22.949687 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.940149    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:00:22.949687 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.940209    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8" podUID="47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:00:22.949687 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.947931    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:00:22.949687 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.947986    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks" podUID="94fb0221-0a90-421d-9504-80067ac621ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:00:22.963681 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.963644    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:00:22.963934 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:22.963909    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l" podUID="76337bdf-89a0-4e5d-9cb0-437962f9dc8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:00:23.056771 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:23.056713    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:23.223684 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:23.209486    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:23.303913 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:23.303865    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:23.500650 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:23.500526    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:23.661750 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:23.661704    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:23.682140 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:23.662035    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 19:00:23.719640 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:23.662067    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 19:00:23.719640 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:23.662095    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:23.749630 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:23.740765    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 19:00:23.896781 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:23.896744    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:00:23.927796 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:23.927735    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv" podUID="71bee142-6883-471f-933d-18f2424a1bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:00:23.983803 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:23.983740    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:00:23.984185 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:23.984145    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8" podUID="47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:00:23.984430 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:23.984401    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:00:23.984559 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:23.984541    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks" podUID="94fb0221-0a90-421d-9504-80067ac621ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:00:23.985639 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:23.985594    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:00:24.063958 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.063920    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l" podUID="76337bdf-89a0-4e5d-9cb0-437962f9dc8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:00:24.064321 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.064279    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": context deadline exceeded"
Oct 30 19:00:24.286691 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.238288    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:24.428630 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.427872    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 19:00:24.467731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.467673    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 19:00:24.541276 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.477793    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 19:00:24.543955 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.543928    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 19:00:24.735107 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.735066    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 19:00:24.757071 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.757032    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:24.762854 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.762804    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:24.802683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.801540    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 19:00:24.891813 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.891691    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:00:24.891813 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.891750    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv" podUID="71bee142-6883-471f-933d-18f2424a1bac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:00:24.904855 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.904747    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:24.951226 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.951190    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:00:24.951490 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.951457    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8" podUID="47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:00:24.964625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.956912    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:00:24.964625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.956954    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks" podUID="94fb0221-0a90-421d-9504-80067ac621ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:00:24.964625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.959702    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:00:24.964625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:24.959738    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l" podUID="76337bdf-89a0-4e5d-9cb0-437962f9dc8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:00:25.111683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:25.111586    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 19:00:25.134327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:25.124828    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 19:00:25.134327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:25.131728    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 19:00:25.313622 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:25.296136    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 19:00:25.890143 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:25.890096    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:25.946262 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:25.937352    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:25.946262 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:25.937638    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:00:25.946262 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:25.937694    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8" podUID="47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:00:25.946629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:25.946275    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:25.947448 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:25.947425    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:00:25.947690 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:25.947648    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks" podUID="94fb0221-0a90-421d-9504-80067ac621ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:00:25.954234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:25.954199    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:00:25.954416 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:25.954392    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l" podUID="76337bdf-89a0-4e5d-9cb0-437962f9dc8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:00:25.954660 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:25.954643    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:26.330929 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:26.330888    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:26.636674 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:26.636559    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:00:26.972760 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:26.972670    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:26.976594 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:26.976570    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:00:27.058274 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:27.058242    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:27.062849 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:27.062823    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:00:27.292271 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:27.292234    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:27.295269 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:27.295246    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:00:35.532690 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:00:35.532654    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:00:46.696757 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:46.696699    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:46.700108 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:46.700073    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:46.701031 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:46.701006    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:47.139627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:47.138685    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:47.147628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:47.144685    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:47.213742 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:47.213688    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 19:00:47.250749 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:47.250700    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:47.265012 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:47.264975    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:47.537001 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:47.536958    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:47.712992 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:47.712948    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:47.715024 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:47.715002    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 19:00:47.716066 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:47.713539    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:47.725623 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:47.713584    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:47.726143 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:47.726121    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 19:00:47.726313 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:47.726300    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 19:00:48.335629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:48.148668    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:48.362715 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:48.154643    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:48.363685 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:48.220372    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 19:00:48.364058 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:48.363987    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 19:00:48.364058 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:48.364057    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 19:00:48.364212 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:48.364090    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 19:00:48.365363 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:48.359078    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:48.478644 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:48.452739    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 19:00:48.560272 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:48.560228    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:48.573736 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:48.573702    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 19:00:48.731076 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:48.730955    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/health\": context deadline exceeded"
Oct 30 19:00:48.872661 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:48.751615    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 19:00:48.897650 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:48.747805    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:00:48.897650 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:48.790196    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": context deadline exceeded"
Oct 30 19:00:49.041628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.024703    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:49.136752 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.136711    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:00:49.137103 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.137075    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2" podUID="64da4d53-629c-4015-aaf9-2e3765f9dc5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:49.437693 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.371061    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:49.694321 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.371293    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 19:00:49.695006 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.371370    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 19:00:49.720630 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.476195    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 30 19:00:49.758231 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.648861    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:49.897630 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.882895    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:49.954686 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.890026    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 19:00:49.954686 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.898451    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:00:49.954686 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.913656    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:49.954686 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.912128    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:00:49.954686 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.932698    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:00:49.966911 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.966873    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:00:49.967194 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.967161    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj" podUID="c22154c8-6939-496a-b422-29a775dcbf57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:49.967367 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.967335    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7" podUID="48229a00-0acf-4c2b-b14e-d0151692b2d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:49.967522 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.967501    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk" podUID="13a222a4-d62a-4078-bd12-c03a8996dc87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:50.001632 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.997705    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:00:50.001632 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:49.997780    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt" podUID="8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:50.040770 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:50.040720    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6" podUID="f23bf845-e578-4ca7-bc75-898f940fc494" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:50.108758 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:50.108720    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:00:50.109079 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:50.109030    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6" podUID="60c1bed3-9d33-4979-815a-84a7acc09c38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:50.504674 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:50.504632    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:00:50.505256 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:50.505205    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw" podUID="282d4bba-4530-4754-906f-f743b011ab66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:50.705800 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:50.705757    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:50.706503 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:50.706467    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 19:00:50.712888 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:50.712852    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Oct 30 19:00:50.763069 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:50.762968    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:50.767626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:50.764411    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:50.982168 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:50.982126    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": context deadline exceeded"
Oct 30 19:00:51.009762 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:50.982692    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 19:00:51.010280 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:50.984342    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:51.553629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:51.530926    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:51.761629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:51.714001    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:51.917865 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:51.727669    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 19:00:51.945036 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:51.727726    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 19:00:51.986107 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:51.778730    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:52.051170 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:51.778796    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:52.053335 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:52.052177    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:00:52.053335 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:52.052586    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": context deadline exceeded"
Oct 30 19:00:52.053335 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:52.052882    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Oct 30 19:00:52.523000 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:52.522960    4544 patch_prober.go:29] interesting pod/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:00:52.523295 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:52.523263    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk" podUID="0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:52.920681 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:52.917932    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:52.969786 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:52.969730    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:52.989872 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:52.970096    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 19:00:53.066200 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:53.066139    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:53.099902 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:53.089045    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": context deadline exceeded"
Oct 30 19:00:53.099902 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:53.089128    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:53.099902 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:53.092519    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:00:53.099902 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:53.092756    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": context deadline exceeded"
Oct 30 19:00:53.497799 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:53.488015    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:53.701502 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:53.701337    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 19:00:53.702699 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:53.702675    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 19:00:53.702948 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:53.702930    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 19:00:53.703142 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:53.703125    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 19:00:53.709904 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:53.707930    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 19:00:53.709904 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:53.707991    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 19:00:53.709904 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:53.708114    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 19:00:53.714202 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:00:53.714180    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 19:01:05.433844 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:05.433807    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh"]
Oct 30 19:01:05.439406 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:05.439385    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh"
Oct 30 19:01:05.490416 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:05.490384    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh"]
Oct 30 19:01:05.532717 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:05.532687    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-948bc488\""
Oct 30 19:01:05.539885 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:05.539836    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-948bc488-dockercfg-gzt7j\""
Oct 30 19:01:05.569670 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:01:05.555870    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:01:05.633037 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:05.621492    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6235633-b389-4841-ab4b-fdadf12fddd7-plugins-config-volume\") pod \"payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh\" (UID: \"d6235633-b389-4841-ab4b-fdadf12fddd7\") " pod="openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh"
Oct 30 19:01:05.633037 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:05.621555    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4spx\" (UniqueName: \"kubernetes.io/projected/d6235633-b389-4841-ab4b-fdadf12fddd7-kube-api-access-l4spx\") pod \"payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh\" (UID: \"d6235633-b389-4841-ab4b-fdadf12fddd7\") " pod="openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh"
Oct 30 19:01:05.726695 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:05.726082    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6235633-b389-4841-ab4b-fdadf12fddd7-plugins-config-volume\") pod \"payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh\" (UID: \"d6235633-b389-4841-ab4b-fdadf12fddd7\") " pod="openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh"
Oct 30 19:01:05.726695 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:05.726310    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6235633-b389-4841-ab4b-fdadf12fddd7-plugins-config-volume\") pod \"payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh\" (UID: \"d6235633-b389-4841-ab4b-fdadf12fddd7\") " pod="openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh"
Oct 30 19:01:05.726695 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:05.726367    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4spx\" (UniqueName: \"kubernetes.io/projected/d6235633-b389-4841-ab4b-fdadf12fddd7-kube-api-access-l4spx\") pod \"payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh\" (UID: \"d6235633-b389-4841-ab4b-fdadf12fddd7\") " pod="openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh"
Oct 30 19:01:05.783926 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:05.783424    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4spx\" (UniqueName: \"kubernetes.io/projected/d6235633-b389-4841-ab4b-fdadf12fddd7-kube-api-access-l4spx\") pod \"payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh\" (UID: \"d6235633-b389-4841-ab4b-fdadf12fddd7\") " pod="openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh"
Oct 30 19:01:06.055770 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:06.055737    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh"
Oct 30 19:01:06.435920 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:06.435894    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh"]
Oct 30 19:01:06.437328 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:01:06.437302    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6235633_b389_4841_ab4b_fdadf12fddd7.slice/crio-1c31c97115ca22bbc32ab35e506a82e58132d904fd41cd2611a599dc1d1f9f06 WatchSource:0}: Error finding container 1c31c97115ca22bbc32ab35e506a82e58132d904fd41cd2611a599dc1d1f9f06: Status 404 returned error can't find the container with id 1c31c97115ca22bbc32ab35e506a82e58132d904fd41cd2611a599dc1d1f9f06
Oct 30 19:01:07.435152 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:07.435116    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh" event={"ID":"d6235633-b389-4841-ab4b-fdadf12fddd7","Type":"ContainerStarted","Data":"38405bbe7e85cd51fa2a5660ef9bdc0975a09f588e8de14fa0b2edd04564bc87"}
Oct 30 19:01:07.435488 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:07.435468    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh" event={"ID":"d6235633-b389-4841-ab4b-fdadf12fddd7","Type":"ContainerStarted","Data":"1c31c97115ca22bbc32ab35e506a82e58132d904fd41cd2611a599dc1d1f9f06"}
Oct 30 19:01:07.435700 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:07.435686    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh"
Oct 30 19:01:07.543350 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:07.543298    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh" podStartSLOduration=2.543284124 podStartE2EDuration="2.543284124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:07.543275191 +0000 UTC m=+1465.069150287" watchObservedRunningTime="2026-10-30 19:01:07.543284124 +0000 UTC m=+1465.069159219"
Oct 30 19:01:11.275659 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.275595    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"]
Oct 30 19:01:11.280036 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.280016    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.309483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.309456    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndb58\" (UniqueName: \"kubernetes.io/projected/4f494989-34da-4a63-95c2-7440b320bb0a-kube-api-access-ndb58\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.309614 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.309489    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-istio-envoy\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.309614 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.309523    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-workload-socket\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.309701 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.309630    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f494989-34da-4a63-95c2-7440b320bb0a-istio-podinfo\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.309701 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.309670    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-istio-data\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.309701 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.309687    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f494989-34da-4a63-95c2-7440b320bb0a-istio-token\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.309866 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.309842    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f494989-34da-4a63-95c2-7440b320bb0a-istiod-ca-cert\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.310061 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.310038    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-workload-certs\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.310826 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.310126    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-credential-socket\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.386516 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.386466    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"]
Oct 30 19:01:11.412623 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.412252    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-workload-certs\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.412623 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.412325    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-credential-socket\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.412623 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.412517    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndb58\" (UniqueName: \"kubernetes.io/projected/4f494989-34da-4a63-95c2-7440b320bb0a-kube-api-access-ndb58\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.412623 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.412551    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-istio-envoy\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.412927 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.412651    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-workload-socket\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.412927 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.412694    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f494989-34da-4a63-95c2-7440b320bb0a-istio-podinfo\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.412927 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.412732    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-workload-certs\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.412927 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.412751    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-istio-data\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.412927 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.412777    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f494989-34da-4a63-95c2-7440b320bb0a-istio-token\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.412927 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.412841    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f494989-34da-4a63-95c2-7440b320bb0a-istiod-ca-cert\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.414220 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.413317    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-credential-socket\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.414220 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.413829    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-istio-data\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.414220 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.414082    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-workload-socket\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.414220 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.414154    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f494989-34da-4a63-95c2-7440b320bb0a-istiod-ca-cert\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.415518 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.415486    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-istio-envoy\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.417041 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.417013    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f494989-34da-4a63-95c2-7440b320bb0a-istio-podinfo\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.468735 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.468699    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f494989-34da-4a63-95c2-7440b320bb0a-istio-token\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.469136 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.469112    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndb58\" (UniqueName: \"kubernetes.io/projected/4f494989-34da-4a63-95c2-7440b320bb0a-kube-api-access-ndb58\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:11.594170 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:11.594083    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:12.023854 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:12.023826    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"]
Oct 30 19:01:12.025696 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:01:12.025672    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f494989_34da_4a63_95c2_7440b320bb0a.slice/crio-71bbde9d348a6f39fa08c3ebb325ede522be3fefb4f23998a59c03be88fe7418 WatchSource:0}: Error finding container 71bbde9d348a6f39fa08c3ebb325ede522be3fefb4f23998a59c03be88fe7418: Status 404 returned error can't find the container with id 71bbde9d348a6f39fa08c3ebb325ede522be3fefb4f23998a59c03be88fe7418
Oct 30 19:01:12.028069 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:12.028036    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:01:12.028166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:12.028112    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:01:12.028166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:12.028155    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:01:12.474092 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:12.473985    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc" event={"ID":"4f494989-34da-4a63-95c2-7440b320bb0a","Type":"ContainerStarted","Data":"7e935715a3d50e14f720c7a53632b9a421a394e23296ca7de47bfc9b20944871"}
Oct 30 19:01:12.474092 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:12.474091    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc" event={"ID":"4f494989-34da-4a63-95c2-7440b320bb0a","Type":"ContainerStarted","Data":"71bbde9d348a6f39fa08c3ebb325ede522be3fefb4f23998a59c03be88fe7418"}
Oct 30 19:01:12.594720 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:12.594684    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:12.647472 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:12.647419    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc" podStartSLOduration=1.64740331 podStartE2EDuration="1.64740331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:12.643721268 +0000 UTC m=+1470.169596367" watchObservedRunningTime="2026-10-30 19:01:12.64740331 +0000 UTC m=+1470.173278406"
Oct 30 19:01:13.596855 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:13.596727    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:01:13.596855 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:13.596786    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc" podUID="4f494989-34da-4a63-95c2-7440b320bb0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 19:01:14.595196 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:14.595160    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:01:14.595396 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:14.595224    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc" podUID="4f494989-34da-4a63-95c2-7440b320bb0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 19:01:15.615627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:15.613502    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:01:15.615627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:15.613567    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc" podUID="4f494989-34da-4a63-95c2-7440b320bb0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 19:01:16.596140 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:16.596093    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:01:16.596325 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:16.596172    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc" podUID="4f494989-34da-4a63-95c2-7440b320bb0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 19:01:17.596146 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:17.596114    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:01:17.596688 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:17.596656    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc" podUID="4f494989-34da-4a63-95c2-7440b320bb0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 19:01:18.444374 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:18.444345    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh"
Oct 30 19:01:18.596094 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:18.595646    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:01:18.600215 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:18.600155    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc" podUID="4f494989-34da-4a63-95c2-7440b320bb0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 19:01:19.599483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:19.599450    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:19.599904 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:19.599880    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:19.600717 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:19.600697    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:01:23.701272 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:23.701177    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:23.702174 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:23.701831    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:24.098731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:24.098583    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:24.962627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:24.960230    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:25.003627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:24.976931    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"]
Oct 30 19:01:25.003627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:24.994901    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.114635 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.104042    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"]
Oct 30 19:01:25.312827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.112864    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-15046cf2-openshift-default-dockercfg-gtr8k\""
Oct 30 19:01:25.313061 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.134435    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-token\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.313232 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.313209    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-credential-socket\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.320614 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.313415    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istiod-ca-cert\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.320910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.320880    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-envoy\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.321395 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.321267    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-data\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.322318 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.321633    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlgrh\" (UniqueName: \"kubernetes.io/projected/60cb440b-c9ac-4c87-87e8-d66e24e7751d-kube-api-access-hlgrh\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.326680 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.326654    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-podinfo\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.326879 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.326861    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-workload-socket\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.330775 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.330752    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-workload-certs\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.450629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.448530    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-token\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.450629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.448588    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-credential-socket\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.450629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.448701    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istiod-ca-cert\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.450629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.448757    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-envoy\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.450629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.448809    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-data\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.450629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.448883    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlgrh\" (UniqueName: \"kubernetes.io/projected/60cb440b-c9ac-4c87-87e8-d66e24e7751d-kube-api-access-hlgrh\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.450629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.448907    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-podinfo\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.450629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.448976    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-workload-socket\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.450629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.449023    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-workload-certs\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.450629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.449536    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-workload-certs\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.450629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.450247    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-credential-socket\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.451332 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.450896    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istiod-ca-cert\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.465626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.451884    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-data\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.465626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.453407    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-workload-socket\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.465626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.464719    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-podinfo\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.480466 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.478118    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-envoy\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.523801 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.520149    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-token\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.538042 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.529342    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlgrh\" (UniqueName: \"kubernetes.io/projected/60cb440b-c9ac-4c87-87e8-d66e24e7751d-kube-api-access-hlgrh\") pod \"e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") " pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:25.605639 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:25.605496    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:26.046216 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:26.046186    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"]
Oct 30 19:01:26.046788 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:01:26.046760    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60cb440b_c9ac_4c87_87e8_d66e24e7751d.slice/crio-60b9630eda9a70565bae7a445b9b2732e2a0bb81bd318b68a6c9598dcd6105dc WatchSource:0}: Error finding container 60b9630eda9a70565bae7a445b9b2732e2a0bb81bd318b68a6c9598dcd6105dc: Status 404 returned error can't find the container with id 60b9630eda9a70565bae7a445b9b2732e2a0bb81bd318b68a6c9598dcd6105dc
Oct 30 19:01:26.049422 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:26.049389    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:01:26.049513 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:26.049454    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:01:26.049513 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:26.049484    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:01:26.799410 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:26.799372    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf" event={"ID":"60cb440b-c9ac-4c87-87e8-d66e24e7751d","Type":"ContainerStarted","Data":"2e1caeffdfdf1ca4da0244cfdc4828c69da7709e742260d20cee84750b722ac0"}
Oct 30 19:01:26.799595 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:26.799435    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf" event={"ID":"60cb440b-c9ac-4c87-87e8-d66e24e7751d","Type":"ContainerStarted","Data":"60b9630eda9a70565bae7a445b9b2732e2a0bb81bd318b68a6c9598dcd6105dc"}
Oct 30 19:01:27.012063 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:27.012007    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf" podStartSLOduration=3.011993298 podStartE2EDuration="3.011993298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:26.963259952 +0000 UTC m=+1484.489135050" watchObservedRunningTime="2026-10-30 19:01:27.011993298 +0000 UTC m=+1484.537868394"
Oct 30 19:01:27.606003 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:27.605963    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:27.611877 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:27.611851    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:27.807441 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:27.807403    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:27.808473 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:27.808455    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:01:32.646394 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:32.646356    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf"]
Oct 30 19:01:32.652819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:32.652792    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf"
Oct 30 19:01:32.734254 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:32.734208    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf"]
Oct 30 19:01:32.790811 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:32.790777    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-15046cf2\""
Oct 30 19:01:32.791193 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:32.791158    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-15046cf2-dockercfg-p75n8\""
Oct 30 19:01:32.820856 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:32.820818    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf447a67-22ee-4e00-8076-75590ee9cd98-plugins-config-volume\") pod \"payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf\" (UID: \"bf447a67-22ee-4e00-8076-75590ee9cd98\") " pod="openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf"
Oct 30 19:01:32.821078 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:32.821051    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnlfh\" (UniqueName: \"kubernetes.io/projected/bf447a67-22ee-4e00-8076-75590ee9cd98-kube-api-access-cnlfh\") pod \"payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf\" (UID: \"bf447a67-22ee-4e00-8076-75590ee9cd98\") " pod="openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf"
Oct 30 19:01:32.931698 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:32.926644    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf447a67-22ee-4e00-8076-75590ee9cd98-plugins-config-volume\") pod \"payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf\" (UID: \"bf447a67-22ee-4e00-8076-75590ee9cd98\") " pod="openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf"
Oct 30 19:01:32.931698 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:32.926815    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnlfh\" (UniqueName: \"kubernetes.io/projected/bf447a67-22ee-4e00-8076-75590ee9cd98-kube-api-access-cnlfh\") pod \"payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf\" (UID: \"bf447a67-22ee-4e00-8076-75590ee9cd98\") " pod="openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf"
Oct 30 19:01:32.931698 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:32.928255    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf447a67-22ee-4e00-8076-75590ee9cd98-plugins-config-volume\") pod \"payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf\" (UID: \"bf447a67-22ee-4e00-8076-75590ee9cd98\") " pod="openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf"
Oct 30 19:01:32.982216 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:32.982183    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnlfh\" (UniqueName: \"kubernetes.io/projected/bf447a67-22ee-4e00-8076-75590ee9cd98-kube-api-access-cnlfh\") pod \"payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf\" (UID: \"bf447a67-22ee-4e00-8076-75590ee9cd98\") " pod="openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf"
Oct 30 19:01:32.994123 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:32.994092    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf"
Oct 30 19:01:33.370261 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:33.370232    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf"]
Oct 30 19:01:33.370491 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:01:33.370472    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf447a67_22ee_4e00_8076_75590ee9cd98.slice/crio-1bb3b32e99f8f11f3a04c88790213eace2eb8d85dc1750b76d479ec7a40d5205 WatchSource:0}: Error finding container 1bb3b32e99f8f11f3a04c88790213eace2eb8d85dc1750b76d479ec7a40d5205: Status 404 returned error can't find the container with id 1bb3b32e99f8f11f3a04c88790213eace2eb8d85dc1750b76d479ec7a40d5205
Oct 30 19:01:33.847911 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:33.847877    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf" event={"ID":"bf447a67-22ee-4e00-8076-75590ee9cd98","Type":"ContainerStarted","Data":"39171c07808fb1d3ef204d37dfae27ff12893d8101c67789fb87202dba7622ee"}
Oct 30 19:01:33.848310 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:33.847932    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf" event={"ID":"bf447a67-22ee-4e00-8076-75590ee9cd98","Type":"ContainerStarted","Data":"1bb3b32e99f8f11f3a04c88790213eace2eb8d85dc1750b76d479ec7a40d5205"}
Oct 30 19:01:33.848310 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:33.847955    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf"
Oct 30 19:01:33.960009 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:33.959952    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf" podStartSLOduration=1.9599374200000002 podStartE2EDuration="1.95993742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:33.956357601 +0000 UTC m=+1491.482232700" watchObservedRunningTime="2026-10-30 19:01:33.95993742 +0000 UTC m=+1491.485812520"
Oct 30 19:01:34.787089 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.787055    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"]
Oct 30 19:01:34.792014 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.791986    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.842467 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.842427    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ntm7n\""
Oct 30 19:01:34.842467 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.842444    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 19:01:34.842467 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.842447    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 19:01:34.842807 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.842477    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 19:01:34.851855 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.851825    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.852191 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.851865    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.852191 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.852014    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c82dp\" (UniqueName: \"kubernetes.io/projected/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-kube-api-access-c82dp\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.852191 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.852078    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.852191 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.852148    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.852354 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.852241    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.852354 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.852324    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.894766 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.894730    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"]
Oct 30 19:01:34.953926 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.953886    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.954098 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.954024    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.954192 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.954152    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.954300 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.954278    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.954463 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.954440    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.954582 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.954562    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c82dp\" (UniqueName: \"kubernetes.io/projected/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-kube-api-access-c82dp\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.954696 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.954677    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.955371 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.955345    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.956677 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.956647    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.956782 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.956755    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.956878 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.956855    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:34.958459 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:34.958438    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:35.010175 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:35.010138    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:35.010175 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:35.010143    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c82dp\" (UniqueName: \"kubernetes.io/projected/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-kube-api-access-c82dp\") pod \"istiod-openshift-gateway-7ddd9cfdc8-776l6\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:35.105378 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:35.105278    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:35.528055 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:35.528021    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"]
Oct 30 19:01:35.533077 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:01:35.533037    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe8e5ac9_0b0a_4a19_868e_47968373cb8d.slice/crio-cb04a172115ffe5554d701e87bcebd292018890bf6844308e1efaa95599bb166 WatchSource:0}: Error finding container cb04a172115ffe5554d701e87bcebd292018890bf6844308e1efaa95599bb166: Status 404 returned error can't find the container with id cb04a172115ffe5554d701e87bcebd292018890bf6844308e1efaa95599bb166
Oct 30 19:01:35.535885 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:01:35.535859    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:01:35.536169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:35.536133    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:01:35.536214 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:35.536202    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:01:35.862138 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:35.862042    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6" event={"ID":"fe8e5ac9-0b0a-4a19-868e-47968373cb8d","Type":"ContainerStarted","Data":"711fe82075a839a6e7634345f40d6d5bca81a9315a3d737d0d62eb263894923c"}
Oct 30 19:01:35.862138 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:35.862122    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6" event={"ID":"fe8e5ac9-0b0a-4a19-868e-47968373cb8d","Type":"ContainerStarted","Data":"cb04a172115ffe5554d701e87bcebd292018890bf6844308e1efaa95599bb166"}
Oct 30 19:01:35.862663 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:35.862194    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:35.995311 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:35.995263    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6" podStartSLOduration=1.995248139 podStartE2EDuration="1.995248139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:35.991494935 +0000 UTC m=+1493.517370035" watchObservedRunningTime="2026-10-30 19:01:35.995248139 +0000 UTC m=+1493.521123235"
Oct 30 19:01:36.874447 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:36.874411    4544 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-776l6 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:01:36.875110 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:36.875079    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6" podUID="fe8e5ac9-0b0a-4a19-868e-47968373cb8d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:01:39.872214 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:39.872181    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:01:42.793266 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:42.793233    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 19:01:42.796107 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:42.796085    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 19:01:44.858870 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:01:44.858841    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf"
Oct 30 19:02:05.532776 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:02:05.532738    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:02:35.532237 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:02:35.532199    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:02:45.897628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:45.897008    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": context deadline exceeded"
Oct 30 19:02:46.013767 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:46.013715    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:46.088691 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:46.087693    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:46.231795 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:46.231692    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/health\": context deadline exceeded"
Oct 30 19:02:46.703751 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:46.703699    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:46.704362 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:46.704324    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 19:02:46.705957 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:46.705929    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:46.775627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:46.774140    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:46.887734 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:46.887684    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:46.897723 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:46.897680    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:46.898815 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:46.898790    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 19:02:47.007870 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:47.007771    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:02:47.008260 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:47.008203    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75" podUID="be77624e-eeb7-44f4-8d9e-dfd58897e555" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:47.019742 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:47.019663    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:47.036380 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:47.020161    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 19:02:47.284912 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:47.267485    4544 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:47.284912 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:47.270688    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:47.317622 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:47.295203    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 19:02:47.733907 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:47.733787    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:47.735819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:47.735781    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": context deadline exceeded"
Oct 30 19:02:47.737432 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:47.737412    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 19:02:47.763793 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:47.762414    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 19:02:47.775849 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:47.775801    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:47.778546 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:47.777538    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 19:02:47.851697 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:47.851668    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 19:02:47.899757 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:47.899709    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:47.901652 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:47.901598    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:47.918678 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:47.913354    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 19:02:47.918678 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:47.913806    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 19:02:48.079627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:48.065220    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 19:02:48.079627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:48.065281    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 19:02:48.079627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:48.065314    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 19:02:48.123742 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:48.123689    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 19:02:48.129485 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:48.127914    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 19:02:55.874575 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:55.874495    4544 generic.go:356] "Generic (PLEG): container finished" podID="60cb440b-c9ac-4c87-87e8-d66e24e7751d" containerID="2e1caeffdfdf1ca4da0244cfdc4828c69da7709e742260d20cee84750b722ac0" exitCode=0
Oct 30 19:02:55.875042 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:55.874569    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf" event={"ID":"60cb440b-c9ac-4c87-87e8-d66e24e7751d","Type":"ContainerDied","Data":"2e1caeffdfdf1ca4da0244cfdc4828c69da7709e742260d20cee84750b722ac0"}
Oct 30 19:02:55.875096 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:55.875046    4544 scope.go:122] "RemoveContainer" containerID="2e1caeffdfdf1ca4da0244cfdc4828c69da7709e742260d20cee84750b722ac0"
Oct 30 19:02:55.876900 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:55.876870    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:02:55.877016 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:55.876944    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:02:55.877016 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:55.876979    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:02:56.885774 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:56.885729    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf" event={"ID":"60cb440b-c9ac-4c87-87e8-d66e24e7751d","Type":"ContainerStarted","Data":"c948aac3e5337cbeafd90be127675e48374707d791b02e1c67374ed0a557c589"}
Oct 30 19:02:57.606375 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:57.606313    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:02:57.607423 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:57.607298    4544 patch_prober.go:29] interesting pod/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:02:57.607423 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:57.607349    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf" podUID="60cb440b-c9ac-4c87-87e8-d66e24e7751d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:02:57.808038 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:57.808002    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:02:58.606693 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:58.606661    4544 patch_prober.go:29] interesting pod/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:02:58.607109 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:58.606722    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf" podUID="60cb440b-c9ac-4c87-87e8-d66e24e7751d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:02:59.606832 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:59.606797    4544 patch_prober.go:29] interesting pod/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:02:59.607243 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:02:59.606852    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf" podUID="60cb440b-c9ac-4c87-87e8-d66e24e7751d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:03:00.606594 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:00.606190    4544 patch_prober.go:29] interesting pod/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:03:00.606594 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:00.606253    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf" podUID="60cb440b-c9ac-4c87-87e8-d66e24e7751d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:03:00.670812 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:00.670774    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh"]
Oct 30 19:03:01.606229 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:01.606191    4544 patch_prober.go:29] interesting pod/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:03:01.606400 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:01.606248    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf" podUID="60cb440b-c9ac-4c87-87e8-d66e24e7751d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:03:02.641964 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:02.640585    4544 patch_prober.go:29] interesting pod/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:03:02.641964 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:02.640712    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf" podUID="60cb440b-c9ac-4c87-87e8-d66e24e7751d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:03:03.610216 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:03.610188    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:03:03.611350 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:03.611328    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:03:05.533573 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:03:05.533531    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:03:05.671994 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:05.671930    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh" podUID="d6235633-b389-4841-ab4b-fdadf12fddd7" containerName="payload-processing" containerID="cri-o://38405bbe7e85cd51fa2a5660ef9bdc0975a09f588e8de14fa0b2edd04564bc87" gracePeriod=25
Oct 30 19:03:05.941436 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:05.941406    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh"
Oct 30 19:03:05.958251 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:05.958221    4544 generic.go:356] "Generic (PLEG): container finished" podID="d6235633-b389-4841-ab4b-fdadf12fddd7" containerID="38405bbe7e85cd51fa2a5660ef9bdc0975a09f588e8de14fa0b2edd04564bc87" exitCode=0
Oct 30 19:03:05.958382 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:05.958266    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh"
Oct 30 19:03:05.958382 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:05.958297    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh" event={"ID":"d6235633-b389-4841-ab4b-fdadf12fddd7","Type":"ContainerDied","Data":"38405bbe7e85cd51fa2a5660ef9bdc0975a09f588e8de14fa0b2edd04564bc87"}
Oct 30 19:03:05.958382 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:05.958376    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh" event={"ID":"d6235633-b389-4841-ab4b-fdadf12fddd7","Type":"ContainerDied","Data":"1c31c97115ca22bbc32ab35e506a82e58132d904fd41cd2611a599dc1d1f9f06"}
Oct 30 19:03:05.958489 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:05.958400    4544 scope.go:122] "RemoveContainer" containerID="38405bbe7e85cd51fa2a5660ef9bdc0975a09f588e8de14fa0b2edd04564bc87"
Oct 30 19:03:05.968768 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:05.968753    4544 scope.go:122] "RemoveContainer" containerID="38405bbe7e85cd51fa2a5660ef9bdc0975a09f588e8de14fa0b2edd04564bc87"
Oct 30 19:03:05.969032 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:03:05.969012    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38405bbe7e85cd51fa2a5660ef9bdc0975a09f588e8de14fa0b2edd04564bc87\": container with ID starting with 38405bbe7e85cd51fa2a5660ef9bdc0975a09f588e8de14fa0b2edd04564bc87 not found: ID does not exist" containerID="38405bbe7e85cd51fa2a5660ef9bdc0975a09f588e8de14fa0b2edd04564bc87"
Oct 30 19:03:05.969079 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:05.969041    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38405bbe7e85cd51fa2a5660ef9bdc0975a09f588e8de14fa0b2edd04564bc87"} err="failed to get container status \"38405bbe7e85cd51fa2a5660ef9bdc0975a09f588e8de14fa0b2edd04564bc87\": rpc error: code = NotFound desc = could not find container \"38405bbe7e85cd51fa2a5660ef9bdc0975a09f588e8de14fa0b2edd04564bc87\": container with ID starting with 38405bbe7e85cd51fa2a5660ef9bdc0975a09f588e8de14fa0b2edd04564bc87 not found: ID does not exist"
Oct 30 19:03:06.092193 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:06.092107    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6235633-b389-4841-ab4b-fdadf12fddd7-kube-api-access-l4spx\" (UniqueName: \"kubernetes.io/projected/d6235633-b389-4841-ab4b-fdadf12fddd7-kube-api-access-l4spx\") pod \"d6235633-b389-4841-ab4b-fdadf12fddd7\" (UID: \"d6235633-b389-4841-ab4b-fdadf12fddd7\") "
Oct 30 19:03:06.092363 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:06.092276    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d6235633-b389-4841-ab4b-fdadf12fddd7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6235633-b389-4841-ab4b-fdadf12fddd7-plugins-config-volume\") pod \"d6235633-b389-4841-ab4b-fdadf12fddd7\" (UID: \"d6235633-b389-4841-ab4b-fdadf12fddd7\") "
Oct 30 19:03:06.092727 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:06.092696    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6235633-b389-4841-ab4b-fdadf12fddd7-plugins-config-volume" pod "d6235633-b389-4841-ab4b-fdadf12fddd7" (UID: "d6235633-b389-4841-ab4b-fdadf12fddd7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:03:06.092964 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:06.092943    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6235633-b389-4841-ab4b-fdadf12fddd7-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:03:06.095110 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:06.095082    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6235633-b389-4841-ab4b-fdadf12fddd7-kube-api-access-l4spx" pod "d6235633-b389-4841-ab4b-fdadf12fddd7" (UID: "d6235633-b389-4841-ab4b-fdadf12fddd7"). InnerVolumeSpecName "kube-api-access-l4spx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:06.194887 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:06.194856    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4spx\" (UniqueName: \"kubernetes.io/projected/d6235633-b389-4841-ab4b-fdadf12fddd7-kube-api-access-l4spx\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:03:06.375075 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:06.374997    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh"]
Oct 30 19:03:06.378445 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:06.378417    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-948bc488-585c74b848-nr8zh"]
Oct 30 19:03:06.603272 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:06.603238    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6235633-b389-4841-ab4b-fdadf12fddd7" path="/var/lib/kubelet/pods/d6235633-b389-4841-ab4b-fdadf12fddd7/volumes"
Oct 30 19:03:12.250912 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.250866    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"]
Oct 30 19:03:12.251518 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.251505    4544 cpu_manager.go:414] "containerMap: removing container" podUID="d6235633-b389-4841-ab4b-fdadf12fddd7" containerName="payload-processing"
Oct 30 19:03:12.251580 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.251519    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6235633-b389-4841-ab4b-fdadf12fddd7" containerName="payload-processing"
Oct 30 19:03:12.251661 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.251650    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6235633-b389-4841-ab4b-fdadf12fddd7" containerName="payload-processing"
Oct 30 19:03:12.254923 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.254905    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.339754 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.339722    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-workload-certs\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.339907 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.339851    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-credential-socket\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.339959 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.339903    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b58b94c2-d465-4560-aac1-1393be5c6343-istio-podinfo\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.340044 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.340019    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-workload-socket\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.340177 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.340074    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b58b94c2-d465-4560-aac1-1393be5c6343-istio-token\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.340177 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.340145    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b58b94c2-d465-4560-aac1-1393be5c6343-istiod-ca-cert\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.340287 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.340242    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-istio-data\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.340402 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.340383    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-istio-envoy\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.340461 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.340447    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44qrb\" (UniqueName: \"kubernetes.io/projected/b58b94c2-d465-4560-aac1-1393be5c6343-kube-api-access-44qrb\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.442381 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.442347    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-workload-socket\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.442555 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.442393    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b58b94c2-d465-4560-aac1-1393be5c6343-istio-token\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.442555 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.442429    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b58b94c2-d465-4560-aac1-1393be5c6343-istiod-ca-cert\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.442555 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.442531    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-istio-data\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.442790 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.442726    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-istio-envoy\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.442851 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.442805    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44qrb\" (UniqueName: \"kubernetes.io/projected/b58b94c2-d465-4560-aac1-1393be5c6343-kube-api-access-44qrb\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.442851 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.442831    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-workload-socket\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.442961 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.442850    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-workload-certs\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.443018 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.442976    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-credential-socket\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.443200 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.443172    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-workload-certs\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.443346 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.443322    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-credential-socket\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.443427 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.443338    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-istio-data\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.443427 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.443413    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b58b94c2-d465-4560-aac1-1393be5c6343-istio-podinfo\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.443856 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.443829    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b58b94c2-d465-4560-aac1-1393be5c6343-istiod-ca-cert\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.445377 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.445356    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-istio-envoy\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.446332 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.446299    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b58b94c2-d465-4560-aac1-1393be5c6343-istio-podinfo\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.515072 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.500068    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b58b94c2-d465-4560-aac1-1393be5c6343-istio-token\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.515072 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.500428    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44qrb\" (UniqueName: \"kubernetes.io/projected/b58b94c2-d465-4560-aac1-1393be5c6343-kube-api-access-44qrb\") pod \"e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") " pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:12.515072 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.501643    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"]
Oct 30 19:03:12.568811 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:12.568778    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:13.000483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:13.000443    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"]
Oct 30 19:03:13.046263 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:03:13.045784    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb58b94c2_d465_4560_aac1_1393be5c6343.slice/crio-bbf47ad549363129eedd24a9dcd642438a7511916096b82b5b0ef02d6b0074e9 WatchSource:0}: Error finding container bbf47ad549363129eedd24a9dcd642438a7511916096b82b5b0ef02d6b0074e9: Status 404 returned error can't find the container with id bbf47ad549363129eedd24a9dcd642438a7511916096b82b5b0ef02d6b0074e9
Oct 30 19:03:13.053708 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:13.053664    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:03:13.053825 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:13.053760    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:03:13.053825 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:13.053804    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:03:13.144957 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:13.144920    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n" event={"ID":"b58b94c2-d465-4560-aac1-1393be5c6343","Type":"ContainerStarted","Data":"f812aaf9a69d22ff1c9977a73e78afbbbfb8adde6eb754ced8bdf21f5edb830b"}
Oct 30 19:03:13.145083 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:13.144982    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n" event={"ID":"b58b94c2-d465-4560-aac1-1393be5c6343","Type":"ContainerStarted","Data":"bbf47ad549363129eedd24a9dcd642438a7511916096b82b5b0ef02d6b0074e9"}
Oct 30 19:03:13.263241 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:13.262948    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n" podStartSLOduration=1.262929589 podStartE2EDuration="1.262929589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:13.25508177 +0000 UTC m=+1590.780956866" watchObservedRunningTime="2026-10-30 19:03:13.262929589 +0000 UTC m=+1590.788804689"
Oct 30 19:03:13.569240 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:13.569210    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:14.581704 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:14.574502    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:03:14.581704 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:14.574570    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n" podUID="b58b94c2-d465-4560-aac1-1393be5c6343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 19:03:15.569948 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:15.569904    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:03:15.570157 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:15.569968    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n" podUID="b58b94c2-d465-4560-aac1-1393be5c6343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 19:03:16.575624 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:16.575433    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:03:16.575624 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:16.575496    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n" podUID="b58b94c2-d465-4560-aac1-1393be5c6343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 19:03:17.575626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:17.575065    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:03:17.575626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:17.575133    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n" podUID="b58b94c2-d465-4560-aac1-1393be5c6343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 19:03:18.582627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:18.582016    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:03:18.582627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:18.582076    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n" podUID="b58b94c2-d465-4560-aac1-1393be5c6343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 19:03:19.569625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:19.569157    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:03:19.569625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:19.569229    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n" podUID="b58b94c2-d465-4560-aac1-1393be5c6343" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 19:03:20.737660 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:20.737624    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:20.738274 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:20.738213    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:20.739019 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:20.739003    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:03:35.538526 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:03:35.538487    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:03:38.671296 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:38.671247    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf"]
Oct 30 19:03:43.673152 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:43.673087    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf" podUID="bf447a67-22ee-4e00-8076-75590ee9cd98" containerName="payload-processing" containerID="cri-o://39171c07808fb1d3ef204d37dfae27ff12893d8101c67789fb87202dba7622ee" gracePeriod=25
Oct 30 19:03:43.947400 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:43.947375    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf"
Oct 30 19:03:44.074936 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:44.074902    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bf447a67-22ee-4e00-8076-75590ee9cd98-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf447a67-22ee-4e00-8076-75590ee9cd98-plugins-config-volume\") pod \"bf447a67-22ee-4e00-8076-75590ee9cd98\" (UID: \"bf447a67-22ee-4e00-8076-75590ee9cd98\") "
Oct 30 19:03:44.075123 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:44.074976    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf447a67-22ee-4e00-8076-75590ee9cd98-kube-api-access-cnlfh\" (UniqueName: \"kubernetes.io/projected/bf447a67-22ee-4e00-8076-75590ee9cd98-kube-api-access-cnlfh\") pod \"bf447a67-22ee-4e00-8076-75590ee9cd98\" (UID: \"bf447a67-22ee-4e00-8076-75590ee9cd98\") "
Oct 30 19:03:44.075273 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:44.075249    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf447a67-22ee-4e00-8076-75590ee9cd98-plugins-config-volume" pod "bf447a67-22ee-4e00-8076-75590ee9cd98" (UID: "bf447a67-22ee-4e00-8076-75590ee9cd98"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:03:44.075687 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:44.075666    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf447a67-22ee-4e00-8076-75590ee9cd98-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:03:44.077150 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:44.077129    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf447a67-22ee-4e00-8076-75590ee9cd98-kube-api-access-cnlfh" pod "bf447a67-22ee-4e00-8076-75590ee9cd98" (UID: "bf447a67-22ee-4e00-8076-75590ee9cd98"). InnerVolumeSpecName "kube-api-access-cnlfh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:44.177546 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:44.177459    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnlfh\" (UniqueName: \"kubernetes.io/projected/bf447a67-22ee-4e00-8076-75590ee9cd98-kube-api-access-cnlfh\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:03:44.462468 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:44.462373    4544 generic.go:356] "Generic (PLEG): container finished" podID="bf447a67-22ee-4e00-8076-75590ee9cd98" containerID="39171c07808fb1d3ef204d37dfae27ff12893d8101c67789fb87202dba7622ee" exitCode=0
Oct 30 19:03:44.462468 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:44.462440    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf"
Oct 30 19:03:44.462728 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:44.462457    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf" event={"ID":"bf447a67-22ee-4e00-8076-75590ee9cd98","Type":"ContainerDied","Data":"39171c07808fb1d3ef204d37dfae27ff12893d8101c67789fb87202dba7622ee"}
Oct 30 19:03:44.462728 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:44.462559    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf" event={"ID":"bf447a67-22ee-4e00-8076-75590ee9cd98","Type":"ContainerDied","Data":"1bb3b32e99f8f11f3a04c88790213eace2eb8d85dc1750b76d479ec7a40d5205"}
Oct 30 19:03:44.462728 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:44.462591    4544 scope.go:122] "RemoveContainer" containerID="39171c07808fb1d3ef204d37dfae27ff12893d8101c67789fb87202dba7622ee"
Oct 30 19:03:44.484880 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:44.484856    4544 scope.go:122] "RemoveContainer" containerID="39171c07808fb1d3ef204d37dfae27ff12893d8101c67789fb87202dba7622ee"
Oct 30 19:03:44.485422 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:03:44.485392    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39171c07808fb1d3ef204d37dfae27ff12893d8101c67789fb87202dba7622ee\": container with ID starting with 39171c07808fb1d3ef204d37dfae27ff12893d8101c67789fb87202dba7622ee not found: ID does not exist" containerID="39171c07808fb1d3ef204d37dfae27ff12893d8101c67789fb87202dba7622ee"
Oct 30 19:03:44.485613 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:44.485569    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39171c07808fb1d3ef204d37dfae27ff12893d8101c67789fb87202dba7622ee"} err="failed to get container status \"39171c07808fb1d3ef204d37dfae27ff12893d8101c67789fb87202dba7622ee\": rpc error: code = NotFound desc = could not find container \"39171c07808fb1d3ef204d37dfae27ff12893d8101c67789fb87202dba7622ee\": container with ID starting with 39171c07808fb1d3ef204d37dfae27ff12893d8101c67789fb87202dba7622ee not found: ID does not exist"
Oct 30 19:03:44.620032 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:44.620001    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf"]
Oct 30 19:03:44.625064 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:44.625029    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-15046cf2-7dfc5cf6b4-4wndf"]
Oct 30 19:03:46.605257 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:03:46.605220    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bf447a67-22ee-4e00-8076-75590ee9cd98" path="/var/lib/kubelet/pods/bf447a67-22ee-4e00-8076-75590ee9cd98/volumes"
Oct 30 19:04:05.533270 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:04:05.533235    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:04:18.771460 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:18.771423    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"]
Oct 30 19:04:18.772118 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:18.771672    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf" podUID="60cb440b-c9ac-4c87-87e8-d66e24e7751d" containerName="istio-proxy" containerID="cri-o://c948aac3e5337cbeafd90be127675e48374707d791b02e1c67374ed0a557c589" gracePeriod=30
Oct 30 19:04:23.991645 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:23.991488    4544 generic.go:356] "Generic (PLEG): container finished" podID="60cb440b-c9ac-4c87-87e8-d66e24e7751d" containerID="c948aac3e5337cbeafd90be127675e48374707d791b02e1c67374ed0a557c589" exitCode=0
Oct 30 19:04:23.991645 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:23.991557    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf" event={"ID":"60cb440b-c9ac-4c87-87e8-d66e24e7751d","Type":"ContainerDied","Data":"c948aac3e5337cbeafd90be127675e48374707d791b02e1c67374ed0a557c589"}
Oct 30 19:04:23.992065 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:23.991657    4544 scope.go:122] "RemoveContainer" containerID="2e1caeffdfdf1ca4da0244cfdc4828c69da7709e742260d20cee84750b722ac0"
Oct 30 19:04:24.071736 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.071710    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:04:24.186675 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.186636    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-token\" (UniqueName: \"kubernetes.io/projected/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-token\") pod \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") "
Oct 30 19:04:24.186869 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.186695    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-workload-socket\") pod \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") "
Oct 30 19:04:24.186869 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.186743    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60cb440b-c9ac-4c87-87e8-d66e24e7751d-kube-api-access-hlgrh\" (UniqueName: \"kubernetes.io/projected/60cb440b-c9ac-4c87-87e8-d66e24e7751d-kube-api-access-hlgrh\") pod \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") "
Oct 30 19:04:24.186869 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.186772    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-workload-certs\") pod \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") "
Oct 30 19:04:24.186869 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.186848    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-envoy\") pod \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") "
Oct 30 19:04:24.187098 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.186994    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-podinfo\") pod \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") "
Oct 30 19:04:24.187098 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.187028    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-workload-socket" pod "60cb440b-c9ac-4c87-87e8-d66e24e7751d" (UID: "60cb440b-c9ac-4c87-87e8-d66e24e7751d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:24.187098 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.187078    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-credential-socket\") pod \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") "
Oct 30 19:04:24.187098 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.187081    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-workload-certs" pod "60cb440b-c9ac-4c87-87e8-d66e24e7751d" (UID: "60cb440b-c9ac-4c87-87e8-d66e24e7751d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:24.187305 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.187103    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istiod-ca-cert\") pod \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") "
Oct 30 19:04:24.187305 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.187159    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-data\") pod \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\" (UID: \"60cb440b-c9ac-4c87-87e8-d66e24e7751d\") "
Oct 30 19:04:24.187403 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.187322    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-credential-socket" pod "60cb440b-c9ac-4c87-87e8-d66e24e7751d" (UID: "60cb440b-c9ac-4c87-87e8-d66e24e7751d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:24.187620 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.187575    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-data" pod "60cb440b-c9ac-4c87-87e8-d66e24e7751d" (UID: "60cb440b-c9ac-4c87-87e8-d66e24e7751d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:24.187737 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.187663    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istiod-ca-cert" pod "60cb440b-c9ac-4c87-87e8-d66e24e7751d" (UID: "60cb440b-c9ac-4c87-87e8-d66e24e7751d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:04:24.188001 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.187980    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:24.188085 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.188008    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:24.188085 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.188024    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:24.188085 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.188039    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:24.188085 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.188052    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:24.189480 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.189455    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-token" pod "60cb440b-c9ac-4c87-87e8-d66e24e7751d" (UID: "60cb440b-c9ac-4c87-87e8-d66e24e7751d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:04:24.189579 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.189462    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-podinfo" pod "60cb440b-c9ac-4c87-87e8-d66e24e7751d" (UID: "60cb440b-c9ac-4c87-87e8-d66e24e7751d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:04:24.189579 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.189495    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60cb440b-c9ac-4c87-87e8-d66e24e7751d-kube-api-access-hlgrh" pod "60cb440b-c9ac-4c87-87e8-d66e24e7751d" (UID: "60cb440b-c9ac-4c87-87e8-d66e24e7751d"). InnerVolumeSpecName "kube-api-access-hlgrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:04:24.189897 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.189874    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-envoy" pod "60cb440b-c9ac-4c87-87e8-d66e24e7751d" (UID: "60cb440b-c9ac-4c87-87e8-d66e24e7751d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:24.290004 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.289971    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:24.290004 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.290002    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlgrh\" (UniqueName: \"kubernetes.io/projected/60cb440b-c9ac-4c87-87e8-d66e24e7751d-kube-api-access-hlgrh\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:24.290204 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.290015    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:24.290204 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.290027    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60cb440b-c9ac-4c87-87e8-d66e24e7751d-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:24.999306 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.999225    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf" event={"ID":"60cb440b-c9ac-4c87-87e8-d66e24e7751d","Type":"ContainerDied","Data":"60b9630eda9a70565bae7a445b9b2732e2a0bb81bd318b68a6c9598dcd6105dc"}
Oct 30 19:04:24.999306 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.999291    4544 scope.go:122] "RemoveContainer" containerID="c948aac3e5337cbeafd90be127675e48374707d791b02e1c67374ed0a557c589"
Oct 30 19:04:24.999802 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:24.999239    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"
Oct 30 19:04:25.203706 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:25.203674    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"]
Oct 30 19:04:25.206901 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:25.206877    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-15046cf2-openshift-default-85cc8f99b7-bdfgf"]
Oct 30 19:04:26.605175 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:26.605137    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60cb440b-c9ac-4c87-87e8-d66e24e7751d" path="/var/lib/kubelet/pods/60cb440b-c9ac-4c87-87e8-d66e24e7751d/volumes"
Oct 30 19:04:35.533927 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:04:35.533888    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:04:41.390419 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:41.390381    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"]
Oct 30 19:04:41.390887 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:41.390644    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb" podUID="34e14ce3-0d46-4301-b047-43b7ddc23088" containerName="istio-proxy" containerID="cri-o://c7573282c64be82db694a1731b657a872eab568a11076d920469fd5a7cac5aae" gracePeriod=30
Oct 30 19:04:41.434875 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:41.434842    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"]
Oct 30 19:04:41.435118 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:41.435092    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df" podUID="599ac606-1d53-445d-a863-9fd9564cc549" containerName="istio-proxy" containerID="cri-o://968e8aa44bc3a01ba9a91a874775e3b73d54b271750d3eddeaeeeb1811eeb4e4" gracePeriod=30
Oct 30 19:04:41.438471 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:41.438445    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"]
Oct 30 19:04:41.438693 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:41.438672    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff" podUID="49a8e4ba-b424-4727-8e8b-23a3eaccece2" containerName="istio-proxy" containerID="cri-o://72f098bcdf198672d76b3899149ab9fbe84db3b45d641040b406a2e5cc6133dc" gracePeriod=30
Oct 30 19:04:41.481546 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:41.481513    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"]
Oct 30 19:04:41.481851 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:41.481824    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq" podUID="0fe6b941-e299-4145-b798-3d590e53048e" containerName="istio-proxy" containerID="cri-o://f596c62e9b83ec5311a9e443db8e9b6ffefc2026d0ff82c2f21ef99c65a21616" gracePeriod=30
Oct 30 19:04:42.917732 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:42.917699    4544 patch_prober.go:29] interesting pod/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:04:42.918114 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:42.917760    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df" podUID="599ac606-1d53-445d-a863-9fd9564cc549" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:04:46.795316 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.795230    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 19:04:46.836655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.834955    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 19:04:46.884355 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.884328    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-token\" (UniqueName: \"kubernetes.io/projected/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-token\") pod \"34e14ce3-0d46-4301-b047-43b7ddc23088\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") "
Oct 30 19:04:46.884495 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.884401    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/599ac606-1d53-445d-a863-9fd9564cc549-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/599ac606-1d53-445d-a863-9fd9564cc549-istiod-ca-cert\") pod \"599ac606-1d53-445d-a863-9fd9564cc549\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") "
Oct 30 19:04:46.884495 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.884422    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-istio-data\") pod \"599ac606-1d53-445d-a863-9fd9564cc549\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") "
Oct 30 19:04:46.884495 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.884447    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34e14ce3-0d46-4301-b047-43b7ddc23088-kube-api-access-ng6k8\" (UniqueName: \"kubernetes.io/projected/34e14ce3-0d46-4301-b047-43b7ddc23088-kube-api-access-ng6k8\") pod \"34e14ce3-0d46-4301-b047-43b7ddc23088\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") "
Oct 30 19:04:46.884495 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.884492    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-podinfo\") pod \"34e14ce3-0d46-4301-b047-43b7ddc23088\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") "
Oct 30 19:04:46.884731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.884521    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-workload-certs\") pod \"599ac606-1d53-445d-a863-9fd9564cc549\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") "
Oct 30 19:04:46.884731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.884573    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-credential-socket\") pod \"599ac606-1d53-445d-a863-9fd9564cc549\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") "
Oct 30 19:04:46.884731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.884632    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/599ac606-1d53-445d-a863-9fd9564cc549-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/599ac606-1d53-445d-a863-9fd9564cc549-istio-podinfo\") pod \"599ac606-1d53-445d-a863-9fd9564cc549\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") "
Oct 30 19:04:46.884731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.884671    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-workload-socket\") pod \"599ac606-1d53-445d-a863-9fd9564cc549\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") "
Oct 30 19:04:46.884731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.884695    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-workload-socket\") pod \"34e14ce3-0d46-4301-b047-43b7ddc23088\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") "
Oct 30 19:04:46.884731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.884724    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/599ac606-1d53-445d-a863-9fd9564cc549-istio-token\" (UniqueName: \"kubernetes.io/projected/599ac606-1d53-445d-a863-9fd9564cc549-istio-token\") pod \"599ac606-1d53-445d-a863-9fd9564cc549\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") "
Oct 30 19:04:46.885035 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.884782    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/599ac606-1d53-445d-a863-9fd9564cc549-kube-api-access-6hj9x\" (UniqueName: \"kubernetes.io/projected/599ac606-1d53-445d-a863-9fd9564cc549-kube-api-access-6hj9x\") pod \"599ac606-1d53-445d-a863-9fd9564cc549\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") "
Oct 30 19:04:46.885035 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.884834    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/599ac606-1d53-445d-a863-9fd9564cc549-istiod-ca-cert" pod "599ac606-1d53-445d-a863-9fd9564cc549" (UID: "599ac606-1d53-445d-a863-9fd9564cc549"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:04:46.885035 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.884905    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-istio-envoy\") pod \"599ac606-1d53-445d-a863-9fd9564cc549\" (UID: \"599ac606-1d53-445d-a863-9fd9564cc549\") "
Oct 30 19:04:46.885035 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.884952    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-workload-certs" pod "599ac606-1d53-445d-a863-9fd9564cc549" (UID: "599ac606-1d53-445d-a863-9fd9564cc549"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.885035 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.884961    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-envoy\") pod \"34e14ce3-0d46-4301-b047-43b7ddc23088\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") "
Oct 30 19:04:46.885035 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.884996    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-workload-socket" pod "599ac606-1d53-445d-a863-9fd9564cc549" (UID: "599ac606-1d53-445d-a863-9fd9564cc549"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.885035 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.885026    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-credential-socket\") pod \"34e14ce3-0d46-4301-b047-43b7ddc23088\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") "
Oct 30 19:04:46.885395 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.885149    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-istio-data" pod "599ac606-1d53-445d-a863-9fd9564cc549" (UID: "599ac606-1d53-445d-a863-9fd9564cc549"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.885395 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.885210    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-workload-socket" pod "34e14ce3-0d46-4301-b047-43b7ddc23088" (UID: "34e14ce3-0d46-4301-b047-43b7ddc23088"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.885395 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.885226    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-credential-socket" pod "599ac606-1d53-445d-a863-9fd9564cc549" (UID: "599ac606-1d53-445d-a863-9fd9564cc549"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.885585 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.885553    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-workload-certs\") pod \"34e14ce3-0d46-4301-b047-43b7ddc23088\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") "
Oct 30 19:04:46.885751 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.885634    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-data\") pod \"34e14ce3-0d46-4301-b047-43b7ddc23088\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") "
Oct 30 19:04:46.885751 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.885688    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/34e14ce3-0d46-4301-b047-43b7ddc23088-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34e14ce3-0d46-4301-b047-43b7ddc23088-istiod-ca-cert\") pod \"34e14ce3-0d46-4301-b047-43b7ddc23088\" (UID: \"34e14ce3-0d46-4301-b047-43b7ddc23088\") "
Oct 30 19:04:46.886478 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.886232    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-credential-socket" pod "34e14ce3-0d46-4301-b047-43b7ddc23088" (UID: "34e14ce3-0d46-4301-b047-43b7ddc23088"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.886556 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.886528    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/599ac606-1d53-445d-a863-9fd9564cc549-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.886556 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.886550    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.886681 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.886564    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.886681 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.886579    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.886681 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.886593    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.886681 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.886624    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.886681 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.886641    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.887033 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.886842    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-workload-certs" pod "34e14ce3-0d46-4301-b047-43b7ddc23088" (UID: "34e14ce3-0d46-4301-b047-43b7ddc23088"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.887033 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.886856    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-data" pod "34e14ce3-0d46-4301-b047-43b7ddc23088" (UID: "34e14ce3-0d46-4301-b047-43b7ddc23088"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.887233 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.887142    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34e14ce3-0d46-4301-b047-43b7ddc23088-istiod-ca-cert" pod "34e14ce3-0d46-4301-b047-43b7ddc23088" (UID: "34e14ce3-0d46-4301-b047-43b7ddc23088"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:04:46.889039 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.889013    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-envoy" pod "34e14ce3-0d46-4301-b047-43b7ddc23088" (UID: "34e14ce3-0d46-4301-b047-43b7ddc23088"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.889773 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.889753    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34e14ce3-0d46-4301-b047-43b7ddc23088-kube-api-access-ng6k8" pod "34e14ce3-0d46-4301-b047-43b7ddc23088" (UID: "34e14ce3-0d46-4301-b047-43b7ddc23088"). InnerVolumeSpecName "kube-api-access-ng6k8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:04:46.890481 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.890456    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/599ac606-1d53-445d-a863-9fd9564cc549-istio-podinfo" pod "599ac606-1d53-445d-a863-9fd9564cc549" (UID: "599ac606-1d53-445d-a863-9fd9564cc549"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:04:46.890665 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.890642    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-token" pod "34e14ce3-0d46-4301-b047-43b7ddc23088" (UID: "34e14ce3-0d46-4301-b047-43b7ddc23088"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:04:46.891000 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.890976    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-podinfo" pod "34e14ce3-0d46-4301-b047-43b7ddc23088" (UID: "34e14ce3-0d46-4301-b047-43b7ddc23088"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:04:46.891409 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.891389    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/599ac606-1d53-445d-a863-9fd9564cc549-istio-token" pod "599ac606-1d53-445d-a863-9fd9564cc549" (UID: "599ac606-1d53-445d-a863-9fd9564cc549"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:04:46.891511 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.891420    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/599ac606-1d53-445d-a863-9fd9564cc549-kube-api-access-6hj9x" pod "599ac606-1d53-445d-a863-9fd9564cc549" (UID: "599ac606-1d53-445d-a863-9fd9564cc549"). InnerVolumeSpecName "kube-api-access-6hj9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:04:46.891976 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.891955    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-istio-envoy" pod "599ac606-1d53-445d-a863-9fd9564cc549" (UID: "599ac606-1d53-445d-a863-9fd9564cc549"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.895419 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.895395    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 19:04:46.900309 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.900288    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 19:04:46.987495 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.987398    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0fe6b941-e299-4145-b798-3d590e53048e-istio-token\" (UniqueName: \"kubernetes.io/projected/0fe6b941-e299-4145-b798-3d590e53048e-istio-token\") pod \"0fe6b941-e299-4145-b798-3d590e53048e\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") "
Oct 30 19:04:46.987495 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.987470    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-token\" (UniqueName: \"kubernetes.io/projected/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-token\") pod \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") "
Oct 30 19:04:46.987761 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.987518    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-workload-socket\") pod \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") "
Oct 30 19:04:46.987761 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.987714    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49a8e4ba-b424-4727-8e8b-23a3eaccece2-kube-api-access-95ktm\" (UniqueName: \"kubernetes.io/projected/49a8e4ba-b424-4727-8e8b-23a3eaccece2-kube-api-access-95ktm\") pod \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") "
Oct 30 19:04:46.987877 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.987761    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-istio-data\") pod \"0fe6b941-e299-4145-b798-3d590e53048e\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") "
Oct 30 19:04:46.987877 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.987831    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-podinfo\") pod \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") "
Oct 30 19:04:46.987995 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.987886    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0fe6b941-e299-4145-b798-3d590e53048e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe6b941-e299-4145-b798-3d590e53048e-istio-podinfo\") pod \"0fe6b941-e299-4145-b798-3d590e53048e\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") "
Oct 30 19:04:46.987995 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.987895    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-workload-socket" pod "49a8e4ba-b424-4727-8e8b-23a3eaccece2" (UID: "49a8e4ba-b424-4727-8e8b-23a3eaccece2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.987995 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.987933    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0fe6b941-e299-4145-b798-3d590e53048e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe6b941-e299-4145-b798-3d590e53048e-istiod-ca-cert\") pod \"0fe6b941-e299-4145-b798-3d590e53048e\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") "
Oct 30 19:04:46.987995 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.987968    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-credential-socket\") pod \"0fe6b941-e299-4145-b798-3d590e53048e\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") "
Oct 30 19:04:46.988177 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.988020    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0fe6b941-e299-4145-b798-3d590e53048e-kube-api-access-9klsg\" (UniqueName: \"kubernetes.io/projected/0fe6b941-e299-4145-b798-3d590e53048e-kube-api-access-9klsg\") pod \"0fe6b941-e299-4145-b798-3d590e53048e\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") "
Oct 30 19:04:46.988177 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.988080    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-workload-certs\") pod \"0fe6b941-e299-4145-b798-3d590e53048e\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") "
Oct 30 19:04:46.988177 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.988148    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-istio-data" pod "0fe6b941-e299-4145-b798-3d590e53048e" (UID: "0fe6b941-e299-4145-b798-3d590e53048e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.988177 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.988153    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-istio-envoy\") pod \"0fe6b941-e299-4145-b798-3d590e53048e\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") "
Oct 30 19:04:46.988382 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.988219    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-workload-certs\") pod \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") "
Oct 30 19:04:46.988382 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.988300    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istiod-ca-cert\") pod \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") "
Oct 30 19:04:46.988382 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.988354    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-data\") pod \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") "
Oct 30 19:04:46.988545 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.988400    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-envoy\") pod \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") "
Oct 30 19:04:46.988545 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.988429    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-credential-socket\") pod \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\" (UID: \"49a8e4ba-b424-4727-8e8b-23a3eaccece2\") "
Oct 30 19:04:46.988545 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.988474    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-workload-socket\") pod \"0fe6b941-e299-4145-b798-3d590e53048e\" (UID: \"0fe6b941-e299-4145-b798-3d590e53048e\") "
Oct 30 19:04:46.989327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.989081    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fe6b941-e299-4145-b798-3d590e53048e-istiod-ca-cert" pod "0fe6b941-e299-4145-b798-3d590e53048e" (UID: "0fe6b941-e299-4145-b798-3d590e53048e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:04:46.989327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.989299    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-workload-socket" pod "0fe6b941-e299-4145-b798-3d590e53048e" (UID: "0fe6b941-e299-4145-b798-3d590e53048e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.989502 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.989474    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-workload-certs" pod "49a8e4ba-b424-4727-8e8b-23a3eaccece2" (UID: "49a8e4ba-b424-4727-8e8b-23a3eaccece2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.989848 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.989782    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istiod-ca-cert" pod "49a8e4ba-b424-4727-8e8b-23a3eaccece2" (UID: "49a8e4ba-b424-4727-8e8b-23a3eaccece2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:04:46.990061 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.990030    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-data" pod "49a8e4ba-b424-4727-8e8b-23a3eaccece2" (UID: "49a8e4ba-b424-4727-8e8b-23a3eaccece2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.990734 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.990706    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-istio-envoy" pod "0fe6b941-e299-4145-b798-3d590e53048e" (UID: "0fe6b941-e299-4145-b798-3d590e53048e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.990734 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.990705    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-token" pod "49a8e4ba-b424-4727-8e8b-23a3eaccece2" (UID: "49a8e4ba-b424-4727-8e8b-23a3eaccece2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:04:46.990961 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.990939    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-workload-certs" pod "0fe6b941-e299-4145-b798-3d590e53048e" (UID: "0fe6b941-e299-4145-b798-3d590e53048e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.991034 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.990953    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-credential-socket" pod "0fe6b941-e299-4145-b798-3d590e53048e" (UID: "0fe6b941-e299-4145-b798-3d590e53048e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.991163 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991138    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991163 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991141    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-credential-socket" pod "49a8e4ba-b424-4727-8e8b-23a3eaccece2" (UID: "49a8e4ba-b424-4727-8e8b-23a3eaccece2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.991289 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991186    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991289 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991204    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991289 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991221    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/599ac606-1d53-445d-a863-9fd9564cc549-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991289 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991237    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991289 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991274    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/599ac606-1d53-445d-a863-9fd9564cc549-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991289 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991288    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991496 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991303    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991496 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991332    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hj9x\" (UniqueName: \"kubernetes.io/projected/599ac606-1d53-445d-a863-9fd9564cc549-kube-api-access-6hj9x\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991496 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991348    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991496 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991363    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991496 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991376    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/599ac606-1d53-445d-a863-9fd9564cc549-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991496 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991390    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991496 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991404    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991496 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991417    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991496 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991431    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991496 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991443    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34e14ce3-0d46-4301-b047-43b7ddc23088-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991496 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991457    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34e14ce3-0d46-4301-b047-43b7ddc23088-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991496 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991470    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe6b941-e299-4145-b798-3d590e53048e-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991496 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991484    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.991496 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.991498    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ng6k8\" (UniqueName: \"kubernetes.io/projected/34e14ce3-0d46-4301-b047-43b7ddc23088-kube-api-access-ng6k8\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:46.993213 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.993088    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0fe6b941-e299-4145-b798-3d590e53048e-istio-podinfo" pod "0fe6b941-e299-4145-b798-3d590e53048e" (UID: "0fe6b941-e299-4145-b798-3d590e53048e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:04:46.993213 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.993188    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fe6b941-e299-4145-b798-3d590e53048e-istio-token" pod "0fe6b941-e299-4145-b798-3d590e53048e" (UID: "0fe6b941-e299-4145-b798-3d590e53048e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:04:46.993361 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.993209    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fe6b941-e299-4145-b798-3d590e53048e-kube-api-access-9klsg" pod "0fe6b941-e299-4145-b798-3d590e53048e" (UID: "0fe6b941-e299-4145-b798-3d590e53048e"). InnerVolumeSpecName "kube-api-access-9klsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:04:46.993361 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.993259    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49a8e4ba-b424-4727-8e8b-23a3eaccece2-kube-api-access-95ktm" pod "49a8e4ba-b424-4727-8e8b-23a3eaccece2" (UID: "49a8e4ba-b424-4727-8e8b-23a3eaccece2"). InnerVolumeSpecName "kube-api-access-95ktm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:04:46.993433 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.993419    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-envoy" pod "49a8e4ba-b424-4727-8e8b-23a3eaccece2" (UID: "49a8e4ba-b424-4727-8e8b-23a3eaccece2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:46.993590 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:46.993574    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-podinfo" pod "49a8e4ba-b424-4727-8e8b-23a3eaccece2" (UID: "49a8e4ba-b424-4727-8e8b-23a3eaccece2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:04:47.092955 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.092916    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe6b941-e299-4145-b798-3d590e53048e-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:47.092955 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.092957    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9klsg\" (UniqueName: \"kubernetes.io/projected/0fe6b941-e299-4145-b798-3d590e53048e-kube-api-access-9klsg\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:47.093221 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.092974    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe6b941-e299-4145-b798-3d590e53048e-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:47.093221 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.092989    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:47.093221 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.093001    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49a8e4ba-b424-4727-8e8b-23a3eaccece2-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:47.093221 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.093011    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe6b941-e299-4145-b798-3d590e53048e-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:47.093221 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.093025    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95ktm\" (UniqueName: \"kubernetes.io/projected/49a8e4ba-b424-4727-8e8b-23a3eaccece2-kube-api-access-95ktm\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:47.093221 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.093037    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49a8e4ba-b424-4727-8e8b-23a3eaccece2-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:47.178458 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.178422    4544 generic.go:356] "Generic (PLEG): container finished" podID="49a8e4ba-b424-4727-8e8b-23a3eaccece2" containerID="72f098bcdf198672d76b3899149ab9fbe84db3b45d641040b406a2e5cc6133dc" exitCode=0
Oct 30 19:04:47.178634 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.178474    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"
Oct 30 19:04:47.178634 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.178504    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff" event={"ID":"49a8e4ba-b424-4727-8e8b-23a3eaccece2","Type":"ContainerDied","Data":"72f098bcdf198672d76b3899149ab9fbe84db3b45d641040b406a2e5cc6133dc"}
Oct 30 19:04:47.178634 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.178564    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff" event={"ID":"49a8e4ba-b424-4727-8e8b-23a3eaccece2","Type":"ContainerDied","Data":"56465488a396008e47af3cffa2f86569f8171ee74f644935482bd590af47f7fc"}
Oct 30 19:04:47.178634 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.178588    4544 scope.go:122] "RemoveContainer" containerID="72f098bcdf198672d76b3899149ab9fbe84db3b45d641040b406a2e5cc6133dc"
Oct 30 19:04:47.180712 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.180689    4544 generic.go:356] "Generic (PLEG): container finished" podID="599ac606-1d53-445d-a863-9fd9564cc549" containerID="968e8aa44bc3a01ba9a91a874775e3b73d54b271750d3eddeaeeeb1811eeb4e4" exitCode=0
Oct 30 19:04:47.180898 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.180780    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"
Oct 30 19:04:47.180898 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.180807    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df" event={"ID":"599ac606-1d53-445d-a863-9fd9564cc549","Type":"ContainerDied","Data":"968e8aa44bc3a01ba9a91a874775e3b73d54b271750d3eddeaeeeb1811eeb4e4"}
Oct 30 19:04:47.180898 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.180852    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df" event={"ID":"599ac606-1d53-445d-a863-9fd9564cc549","Type":"ContainerDied","Data":"0f14a946976e33aa66529bc1c9a1328b8bd4549a146d6a221f8e5d0c9ba6e56b"}
Oct 30 19:04:47.184089 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.184063    4544 generic.go:356] "Generic (PLEG): container finished" podID="34e14ce3-0d46-4301-b047-43b7ddc23088" containerID="c7573282c64be82db694a1731b657a872eab568a11076d920469fd5a7cac5aae" exitCode=0
Oct 30 19:04:47.184243 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.184119    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"
Oct 30 19:04:47.184243 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.184200    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb" event={"ID":"34e14ce3-0d46-4301-b047-43b7ddc23088","Type":"ContainerDied","Data":"c7573282c64be82db694a1731b657a872eab568a11076d920469fd5a7cac5aae"}
Oct 30 19:04:47.184419 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.184245    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb" event={"ID":"34e14ce3-0d46-4301-b047-43b7ddc23088","Type":"ContainerDied","Data":"d839c4119549cd76c8df31e4c0f48224cc427fd29fd00f404ee142af45877b25"}
Oct 30 19:04:47.201643 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.197665    4544 generic.go:356] "Generic (PLEG): container finished" podID="0fe6b941-e299-4145-b798-3d590e53048e" containerID="f596c62e9b83ec5311a9e443db8e9b6ffefc2026d0ff82c2f21ef99c65a21616" exitCode=0
Oct 30 19:04:47.201643 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.197728    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq" event={"ID":"0fe6b941-e299-4145-b798-3d590e53048e","Type":"ContainerDied","Data":"f596c62e9b83ec5311a9e443db8e9b6ffefc2026d0ff82c2f21ef99c65a21616"}
Oct 30 19:04:47.201643 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.197780    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq" event={"ID":"0fe6b941-e299-4145-b798-3d590e53048e","Type":"ContainerDied","Data":"09256580230e3dd00c4932bdae719ab94e15980a492834cff9082a8c88f2a2d0"}
Oct 30 19:04:47.201643 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.197863    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"
Oct 30 19:04:47.207519 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.207500    4544 scope.go:122] "RemoveContainer" containerID="72f098bcdf198672d76b3899149ab9fbe84db3b45d641040b406a2e5cc6133dc"
Oct 30 19:04:47.208427 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:04:47.208395    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72f098bcdf198672d76b3899149ab9fbe84db3b45d641040b406a2e5cc6133dc\": container with ID starting with 72f098bcdf198672d76b3899149ab9fbe84db3b45d641040b406a2e5cc6133dc not found: ID does not exist" containerID="72f098bcdf198672d76b3899149ab9fbe84db3b45d641040b406a2e5cc6133dc"
Oct 30 19:04:47.208842 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.208721    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72f098bcdf198672d76b3899149ab9fbe84db3b45d641040b406a2e5cc6133dc"} err="failed to get container status \"72f098bcdf198672d76b3899149ab9fbe84db3b45d641040b406a2e5cc6133dc\": rpc error: code = NotFound desc = could not find container \"72f098bcdf198672d76b3899149ab9fbe84db3b45d641040b406a2e5cc6133dc\": container with ID starting with 72f098bcdf198672d76b3899149ab9fbe84db3b45d641040b406a2e5cc6133dc not found: ID does not exist"
Oct 30 19:04:47.208953 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.208932    4544 scope.go:122] "RemoveContainer" containerID="968e8aa44bc3a01ba9a91a874775e3b73d54b271750d3eddeaeeeb1811eeb4e4"
Oct 30 19:04:47.220227 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.220206    4544 scope.go:122] "RemoveContainer" containerID="64e54bec5b32ada1c0c7a5d6652d04e4193f6e89a72d0464c3b74e382d3712c8"
Oct 30 19:04:47.229946 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.229924    4544 scope.go:122] "RemoveContainer" containerID="968e8aa44bc3a01ba9a91a874775e3b73d54b271750d3eddeaeeeb1811eeb4e4"
Oct 30 19:04:47.230245 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:04:47.230224    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"968e8aa44bc3a01ba9a91a874775e3b73d54b271750d3eddeaeeeb1811eeb4e4\": container with ID starting with 968e8aa44bc3a01ba9a91a874775e3b73d54b271750d3eddeaeeeb1811eeb4e4 not found: ID does not exist" containerID="968e8aa44bc3a01ba9a91a874775e3b73d54b271750d3eddeaeeeb1811eeb4e4"
Oct 30 19:04:47.230345 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.230250    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"968e8aa44bc3a01ba9a91a874775e3b73d54b271750d3eddeaeeeb1811eeb4e4"} err="failed to get container status \"968e8aa44bc3a01ba9a91a874775e3b73d54b271750d3eddeaeeeb1811eeb4e4\": rpc error: code = NotFound desc = could not find container \"968e8aa44bc3a01ba9a91a874775e3b73d54b271750d3eddeaeeeb1811eeb4e4\": container with ID starting with 968e8aa44bc3a01ba9a91a874775e3b73d54b271750d3eddeaeeeb1811eeb4e4 not found: ID does not exist"
Oct 30 19:04:47.230345 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.230268    4544 scope.go:122] "RemoveContainer" containerID="64e54bec5b32ada1c0c7a5d6652d04e4193f6e89a72d0464c3b74e382d3712c8"
Oct 30 19:04:47.230509 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:04:47.230492    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64e54bec5b32ada1c0c7a5d6652d04e4193f6e89a72d0464c3b74e382d3712c8\": container with ID starting with 64e54bec5b32ada1c0c7a5d6652d04e4193f6e89a72d0464c3b74e382d3712c8 not found: ID does not exist" containerID="64e54bec5b32ada1c0c7a5d6652d04e4193f6e89a72d0464c3b74e382d3712c8"
Oct 30 19:04:47.230582 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.230511    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64e54bec5b32ada1c0c7a5d6652d04e4193f6e89a72d0464c3b74e382d3712c8"} err="failed to get container status \"64e54bec5b32ada1c0c7a5d6652d04e4193f6e89a72d0464c3b74e382d3712c8\": rpc error: code = NotFound desc = could not find container \"64e54bec5b32ada1c0c7a5d6652d04e4193f6e89a72d0464c3b74e382d3712c8\": container with ID starting with 64e54bec5b32ada1c0c7a5d6652d04e4193f6e89a72d0464c3b74e382d3712c8 not found: ID does not exist"
Oct 30 19:04:47.230582 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.230538    4544 scope.go:122] "RemoveContainer" containerID="c7573282c64be82db694a1731b657a872eab568a11076d920469fd5a7cac5aae"
Oct 30 19:04:47.239553 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.239533    4544 scope.go:122] "RemoveContainer" containerID="c7573282c64be82db694a1731b657a872eab568a11076d920469fd5a7cac5aae"
Oct 30 19:04:47.239894 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:04:47.239874    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7573282c64be82db694a1731b657a872eab568a11076d920469fd5a7cac5aae\": container with ID starting with c7573282c64be82db694a1731b657a872eab568a11076d920469fd5a7cac5aae not found: ID does not exist" containerID="c7573282c64be82db694a1731b657a872eab568a11076d920469fd5a7cac5aae"
Oct 30 19:04:47.239952 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.239900    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7573282c64be82db694a1731b657a872eab568a11076d920469fd5a7cac5aae"} err="failed to get container status \"c7573282c64be82db694a1731b657a872eab568a11076d920469fd5a7cac5aae\": rpc error: code = NotFound desc = could not find container \"c7573282c64be82db694a1731b657a872eab568a11076d920469fd5a7cac5aae\": container with ID starting with c7573282c64be82db694a1731b657a872eab568a11076d920469fd5a7cac5aae not found: ID does not exist"
Oct 30 19:04:47.239952 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.239914    4544 scope.go:122] "RemoveContainer" containerID="f596c62e9b83ec5311a9e443db8e9b6ffefc2026d0ff82c2f21ef99c65a21616"
Oct 30 19:04:47.250912 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.250893    4544 scope.go:122] "RemoveContainer" containerID="f596c62e9b83ec5311a9e443db8e9b6ffefc2026d0ff82c2f21ef99c65a21616"
Oct 30 19:04:47.251194 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:04:47.251173    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f596c62e9b83ec5311a9e443db8e9b6ffefc2026d0ff82c2f21ef99c65a21616\": container with ID starting with f596c62e9b83ec5311a9e443db8e9b6ffefc2026d0ff82c2f21ef99c65a21616 not found: ID does not exist" containerID="f596c62e9b83ec5311a9e443db8e9b6ffefc2026d0ff82c2f21ef99c65a21616"
Oct 30 19:04:47.251264 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.251199    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f596c62e9b83ec5311a9e443db8e9b6ffefc2026d0ff82c2f21ef99c65a21616"} err="failed to get container status \"f596c62e9b83ec5311a9e443db8e9b6ffefc2026d0ff82c2f21ef99c65a21616\": rpc error: code = NotFound desc = could not find container \"f596c62e9b83ec5311a9e443db8e9b6ffefc2026d0ff82c2f21ef99c65a21616\": container with ID starting with f596c62e9b83ec5311a9e443db8e9b6ffefc2026d0ff82c2f21ef99c65a21616 not found: ID does not exist"
Oct 30 19:04:47.400775 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.400742    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"]
Oct 30 19:04:47.410407 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.410371    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-4kzcq"]
Oct 30 19:04:47.517738 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.517701    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"]
Oct 30 19:04:47.521201 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.521175    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-tx7df"]
Oct 30 19:04:47.626902 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.626865    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"]
Oct 30 19:04:47.630439 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.630415    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-mktff"]
Oct 30 19:04:47.735465 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.735424    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"]
Oct 30 19:04:47.740063 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:47.740035    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-5f80ac-openshift-default-fffccc485-2ndhb"]
Oct 30 19:04:48.069722 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.069686    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"]
Oct 30 19:04:48.070298 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070283    4544 cpu_manager.go:414] "containerMap: removing container" podUID="0fe6b941-e299-4145-b798-3d590e53048e" containerName="istio-proxy"
Oct 30 19:04:48.070298 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070298    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0fe6b941-e299-4145-b798-3d590e53048e" containerName="istio-proxy"
Oct 30 19:04:48.070424 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070310    4544 cpu_manager.go:414] "containerMap: removing container" podUID="60cb440b-c9ac-4c87-87e8-d66e24e7751d" containerName="istio-proxy"
Oct 30 19:04:48.070424 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070315    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60cb440b-c9ac-4c87-87e8-d66e24e7751d" containerName="istio-proxy"
Oct 30 19:04:48.070424 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070331    4544 cpu_manager.go:414] "containerMap: removing container" podUID="60cb440b-c9ac-4c87-87e8-d66e24e7751d" containerName="istio-proxy"
Oct 30 19:04:48.070424 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070337    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60cb440b-c9ac-4c87-87e8-d66e24e7751d" containerName="istio-proxy"
Oct 30 19:04:48.070424 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070347    4544 cpu_manager.go:414] "containerMap: removing container" podUID="bf447a67-22ee-4e00-8076-75590ee9cd98" containerName="payload-processing"
Oct 30 19:04:48.070424 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070352    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf447a67-22ee-4e00-8076-75590ee9cd98" containerName="payload-processing"
Oct 30 19:04:48.070424 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070362    4544 cpu_manager.go:414] "containerMap: removing container" podUID="599ac606-1d53-445d-a863-9fd9564cc549" containerName="istio-proxy"
Oct 30 19:04:48.070424 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070367    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="599ac606-1d53-445d-a863-9fd9564cc549" containerName="istio-proxy"
Oct 30 19:04:48.070424 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070374    4544 cpu_manager.go:414] "containerMap: removing container" podUID="49a8e4ba-b424-4727-8e8b-23a3eaccece2" containerName="istio-proxy"
Oct 30 19:04:48.070424 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070378    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49a8e4ba-b424-4727-8e8b-23a3eaccece2" containerName="istio-proxy"
Oct 30 19:04:48.070424 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070390    4544 cpu_manager.go:414] "containerMap: removing container" podUID="599ac606-1d53-445d-a863-9fd9564cc549" containerName="istio-proxy"
Oct 30 19:04:48.070424 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070395    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="599ac606-1d53-445d-a863-9fd9564cc549" containerName="istio-proxy"
Oct 30 19:04:48.070424 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070410    4544 cpu_manager.go:414] "containerMap: removing container" podUID="34e14ce3-0d46-4301-b047-43b7ddc23088" containerName="istio-proxy"
Oct 30 19:04:48.070424 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070415    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34e14ce3-0d46-4301-b047-43b7ddc23088" containerName="istio-proxy"
Oct 30 19:04:48.071144 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070494    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="599ac606-1d53-445d-a863-9fd9564cc549" containerName="istio-proxy"
Oct 30 19:04:48.071144 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070505    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="49a8e4ba-b424-4727-8e8b-23a3eaccece2" containerName="istio-proxy"
Oct 30 19:04:48.071144 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070515    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf447a67-22ee-4e00-8076-75590ee9cd98" containerName="payload-processing"
Oct 30 19:04:48.071144 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070525    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="599ac606-1d53-445d-a863-9fd9564cc549" containerName="istio-proxy"
Oct 30 19:04:48.071144 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070531    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="0fe6b941-e299-4145-b798-3d590e53048e" containerName="istio-proxy"
Oct 30 19:04:48.071144 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070538    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="60cb440b-c9ac-4c87-87e8-d66e24e7751d" containerName="istio-proxy"
Oct 30 19:04:48.071144 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070543    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="60cb440b-c9ac-4c87-87e8-d66e24e7751d" containerName="istio-proxy"
Oct 30 19:04:48.071144 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.070549    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="34e14ce3-0d46-4301-b047-43b7ddc23088" containerName="istio-proxy"
Oct 30 19:04:48.075727 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.075702    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:04:48.125459 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.125406    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-4cef683f-serving-cert\""
Oct 30 19:04:48.132622 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.130888    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"]
Oct 30 19:04:48.211571 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.211539    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a84f39-5760-4c04-af47-38ae3fa87a29-openshift-service-ca\") pod \"maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g\" (UID: \"a0a84f39-5760-4c04-af47-38ae3fa87a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:04:48.211856 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.211806    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9snz\" (UniqueName: \"kubernetes.io/projected/a0a84f39-5760-4c04-af47-38ae3fa87a29-kube-api-access-z9snz\") pod \"maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g\" (UID: \"a0a84f39-5760-4c04-af47-38ae3fa87a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:04:48.211939 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.211915    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0a84f39-5760-4c04-af47-38ae3fa87a29-metrics-certs\") pod \"maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g\" (UID: \"a0a84f39-5760-4c04-af47-38ae3fa87a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:04:48.212003 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.211951    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0a84f39-5760-4c04-af47-38ae3fa87a29-maas-api-tls\") pod \"maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g\" (UID: \"a0a84f39-5760-4c04-af47-38ae3fa87a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:04:48.313745 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.313705    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9snz\" (UniqueName: \"kubernetes.io/projected/a0a84f39-5760-4c04-af47-38ae3fa87a29-kube-api-access-z9snz\") pod \"maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g\" (UID: \"a0a84f39-5760-4c04-af47-38ae3fa87a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:04:48.313918 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.313787    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0a84f39-5760-4c04-af47-38ae3fa87a29-metrics-certs\") pod \"maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g\" (UID: \"a0a84f39-5760-4c04-af47-38ae3fa87a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:04:48.313918 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.313815    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0a84f39-5760-4c04-af47-38ae3fa87a29-maas-api-tls\") pod \"maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g\" (UID: \"a0a84f39-5760-4c04-af47-38ae3fa87a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:04:48.314042 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.314023    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a84f39-5760-4c04-af47-38ae3fa87a29-openshift-service-ca\") pod \"maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g\" (UID: \"a0a84f39-5760-4c04-af47-38ae3fa87a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:04:48.314095 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:04:48.314057    4544 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-serving-cert: secret "maas-api-e2e-derive-4cef683f-serving-cert" not found
Oct 30 19:04:48.314155 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:04:48.314148    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0a84f39-5760-4c04-af47-38ae3fa87a29-maas-api-tls podName:a0a84f39-5760-4c04-af47-38ae3fa87a29 nodeName:}" failed. No retries permitted until 2026-10-30 19:04:48.814128347 +0000 UTC m=+1686.340003425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a0a84f39-5760-4c04-af47-38ae3fa87a29-maas-api-tls") pod "maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g" (UID: "a0a84f39-5760-4c04-af47-38ae3fa87a29") : secret "maas-api-e2e-derive-4cef683f-serving-cert" not found
Oct 30 19:04:48.314874 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.314835    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a84f39-5760-4c04-af47-38ae3fa87a29-openshift-service-ca\") pod \"maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g\" (UID: \"a0a84f39-5760-4c04-af47-38ae3fa87a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:04:48.316754 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.316733    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0a84f39-5760-4c04-af47-38ae3fa87a29-metrics-certs\") pod \"maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g\" (UID: \"a0a84f39-5760-4c04-af47-38ae3fa87a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:04:48.368074 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.367991    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9snz\" (UniqueName: \"kubernetes.io/projected/a0a84f39-5760-4c04-af47-38ae3fa87a29-kube-api-access-z9snz\") pod \"maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g\" (UID: \"a0a84f39-5760-4c04-af47-38ae3fa87a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:04:48.607958 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.607920    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0fe6b941-e299-4145-b798-3d590e53048e" path="/var/lib/kubelet/pods/0fe6b941-e299-4145-b798-3d590e53048e/volumes"
Oct 30 19:04:48.608575 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.608525    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34e14ce3-0d46-4301-b047-43b7ddc23088" path="/var/lib/kubelet/pods/34e14ce3-0d46-4301-b047-43b7ddc23088/volumes"
Oct 30 19:04:48.609272 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.609216    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="49a8e4ba-b424-4727-8e8b-23a3eaccece2" path="/var/lib/kubelet/pods/49a8e4ba-b424-4727-8e8b-23a3eaccece2/volumes"
Oct 30 19:04:48.610267 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.610237    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="599ac606-1d53-445d-a863-9fd9564cc549" path="/var/lib/kubelet/pods/599ac606-1d53-445d-a863-9fd9564cc549/volumes"
Oct 30 19:04:48.822873 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.822834    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0a84f39-5760-4c04-af47-38ae3fa87a29-maas-api-tls\") pod \"maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g\" (UID: \"a0a84f39-5760-4c04-af47-38ae3fa87a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:04:48.825325 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.825304    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0a84f39-5760-4c04-af47-38ae3fa87a29-maas-api-tls\") pod \"maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g\" (UID: \"a0a84f39-5760-4c04-af47-38ae3fa87a29\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:04:48.989843 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:48.989812    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:04:49.366873 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:49.366844    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"]
Oct 30 19:04:49.367454 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:04:49.367427    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0a84f39_5760_4c04_af47_38ae3fa87a29.slice/crio-03d1062eb4b8a3ec47a712aea68ad39d298dc926ea6f7ed88ff2a8c022872fd4 WatchSource:0}: Error finding container 03d1062eb4b8a3ec47a712aea68ad39d298dc926ea6f7ed88ff2a8c022872fd4: Status 404 returned error can't find the container with id 03d1062eb4b8a3ec47a712aea68ad39d298dc926ea6f7ed88ff2a8c022872fd4
Oct 30 19:04:49.368978 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:49.368961    4544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:04:50.231156 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:50.231116    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g" event={"ID":"a0a84f39-5760-4c04-af47-38ae3fa87a29","Type":"ContainerStarted","Data":"03d1062eb4b8a3ec47a712aea68ad39d298dc926ea6f7ed88ff2a8c022872fd4"}
Oct 30 19:04:51.247410 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:51.247367    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g" event={"ID":"a0a84f39-5760-4c04-af47-38ae3fa87a29","Type":"ContainerStarted","Data":"f4647b78b4c3cf9b4e0b25c05c56f1b48c03e9bcd979b21c71989d34c62c17a6"}
Oct 30 19:04:51.247896 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:51.247466    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:04:51.357853 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:51.357793    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g" podStartSLOduration=2.491993166 podStartE2EDuration="3.357777667s" totalImagesPullingTime="865.784501ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:51.356283101 +0000 UTC m=+1688.882158191" watchObservedRunningTime="2026-10-30 19:04:51.357777667 +0000 UTC m=+1688.883652763"
Oct 30 19:04:57.261007 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:57.260978    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:04:58.665842 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.665805    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"]
Oct 30 19:04:58.670319 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.670297    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.720640 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.720587    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-314bd6fa-openshift-default-dockercfg-rtl22\""
Oct 30 19:04:58.772913 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.772870    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"]
Oct 30 19:04:58.871013 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.870203    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-istio-data\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.871013 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.870284    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14497a85-b284-431d-930f-a0ac9274ab6c-istiod-ca-cert\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.871013 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.870368    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14497a85-b284-431d-930f-a0ac9274ab6c-istio-podinfo\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.871013 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.870400    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-istio-envoy\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.871013 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.870449    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-workload-certs\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.871013 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.870501    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-credential-socket\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.871013 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.870596    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-workload-socket\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.871013 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.870684    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66ccd\" (UniqueName: \"kubernetes.io/projected/14497a85-b284-431d-930f-a0ac9274ab6c-kube-api-access-66ccd\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.871013 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.870728    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14497a85-b284-431d-930f-a0ac9274ab6c-istio-token\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.972748 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.972651    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14497a85-b284-431d-930f-a0ac9274ab6c-istiod-ca-cert\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.972748 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.972744    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14497a85-b284-431d-930f-a0ac9274ab6c-istio-podinfo\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.972986 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.972768    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-istio-envoy\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.972986 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.972804    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-workload-certs\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.972986 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.972939    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-credential-socket\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.973156 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.973082    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-workload-socket\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.973216 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.973155    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66ccd\" (UniqueName: \"kubernetes.io/projected/14497a85-b284-431d-930f-a0ac9274ab6c-kube-api-access-66ccd\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.973216 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.973199    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14497a85-b284-431d-930f-a0ac9274ab6c-istio-token\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.973337 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.973284    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-credential-socket\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.973337 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.973320    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-istio-data\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.973337 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.973323    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-workload-socket\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.973501 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.973425    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14497a85-b284-431d-930f-a0ac9274ab6c-istiod-ca-cert\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.973566 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.973552    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-istio-data\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.973739 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.973719    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-workload-certs\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.975832 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.975807    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14497a85-b284-431d-930f-a0ac9274ab6c-istio-podinfo\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:58.975992 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:58.975871    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-istio-envoy\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:59.033856 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:59.033822    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66ccd\" (UniqueName: \"kubernetes.io/projected/14497a85-b284-431d-930f-a0ac9274ab6c-kube-api-access-66ccd\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:59.034812 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:59.034779    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14497a85-b284-431d-930f-a0ac9274ab6c-istio-token\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:59.284821 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:59.284785    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:04:59.743331 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:59.743295    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"]
Oct 30 19:04:59.743775 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:04:59.743571    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14497a85_b284_431d_930f_a0ac9274ab6c.slice/crio-24ee10bd7c5cdefe065c6f810da5aa5dce3c4143c0a3389d996381034ce0ab54 WatchSource:0}: Error finding container 24ee10bd7c5cdefe065c6f810da5aa5dce3c4143c0a3389d996381034ce0ab54: Status 404 returned error can't find the container with id 24ee10bd7c5cdefe065c6f810da5aa5dce3c4143c0a3389d996381034ce0ab54
Oct 30 19:04:59.746187 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:59.746156    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:04:59.746262 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:59.746221    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:04:59.746262 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:04:59.746255    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:05:00.313384 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:00.313342    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4" event={"ID":"14497a85-b284-431d-930f-a0ac9274ab6c","Type":"ContainerStarted","Data":"5450673dbd1bdb87f3cdc593241e287b3e8243744a944de9c5e594cb18f1e157"}
Oct 30 19:05:00.313580 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:00.313437    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4" event={"ID":"14497a85-b284-431d-930f-a0ac9274ab6c","Type":"ContainerStarted","Data":"24ee10bd7c5cdefe065c6f810da5aa5dce3c4143c0a3389d996381034ce0ab54"}
Oct 30 19:05:00.479115 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:00.479046    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4" podStartSLOduration=2.479027827 podStartE2EDuration="2.479027827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:00.473866859 +0000 UTC m=+1697.999741979" watchObservedRunningTime="2026-10-30 19:05:00.479027827 +0000 UTC m=+1698.004902924"
Oct 30 19:05:01.286065 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:01.286025    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:05:01.299757 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:01.296881    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:05:01.319580 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:01.319551    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:05:01.320377 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:01.320357    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:05:05.532051 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:05:05.532011    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:05:05.751463 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.751428    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"]
Oct 30 19:05:05.755599 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.755576    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.805237 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.805141    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-g5j8l\""
Oct 30 19:05:05.860623 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.857783    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"]
Oct 30 19:05:05.886784 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.886746    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.887004 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.886925    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wnzd\" (UniqueName: \"kubernetes.io/projected/edd86ca1-466c-4e45-8998-61d7e94fae7d-kube-api-access-9wnzd\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.887139 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.887107    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.887225 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.887205    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.887298 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.887282    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.887357 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.887313    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edd86ca1-466c-4e45-8998-61d7e94fae7d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.887357 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.887337    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.887541 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.887471    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.887651 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.887622    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.989237 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.989193    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wnzd\" (UniqueName: \"kubernetes.io/projected/edd86ca1-466c-4e45-8998-61d7e94fae7d-kube-api-access-9wnzd\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.989432 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.989295    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.989432 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.989351    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.989432 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.989383    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.989432 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.989412    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edd86ca1-466c-4e45-8998-61d7e94fae7d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.989701 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.989442    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.989701 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.989485    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.989701 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.989575    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.989701 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.989645    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.989987 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.989961    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.990180 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.990155    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.990267 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.990182    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.990327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.990273    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edd86ca1-466c-4e45-8998-61d7e94fae7d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.990382 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.990364    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.992179 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.992156    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:05.992179 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:05.992168    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:06.046771 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:06.046729    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:06.046933 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:06.046867    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wnzd\" (UniqueName: \"kubernetes.io/projected/edd86ca1-466c-4e45-8998-61d7e94fae7d-kube-api-access-9wnzd\") pod \"maas-default-gateway-openshift-default-75d7c785f-j76r7\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:06.074478 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:06.074390    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:06.526624 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:06.522185    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"]
Oct 30 19:05:06.554581 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:06.554547    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:05:06.554919 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:06.554646    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:05:06.554919 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:06.554680    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:05:06.720623 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:06.719106    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs"]
Oct 30 19:05:06.780677 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:06.777971    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs"
Oct 30 19:05:06.831044 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:06.831006    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-314bd6fa\""
Oct 30 19:05:06.832115 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:06.832089    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-314bd6fa-dockercfg-f42vb\""
Oct 30 19:05:06.849781 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:06.849694    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs"]
Oct 30 19:05:06.888597 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:06.887746    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv"]
Oct 30 19:05:06.945628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:06.918888    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv"
Oct 30 19:05:06.945628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:06.936886    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8955fdbb-df1e-4a72-83c2-bef3dbb86a68-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs\" (UID: \"8955fdbb-df1e-4a72-83c2-bef3dbb86a68\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs"
Oct 30 19:05:06.945628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:06.937279    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7bmd\" (UniqueName: \"kubernetes.io/projected/8955fdbb-df1e-4a72-83c2-bef3dbb86a68-kube-api-access-n7bmd\") pod \"payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs\" (UID: \"8955fdbb-df1e-4a72-83c2-bef3dbb86a68\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs"
Oct 30 19:05:06.945628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:06.945159    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv"]
Oct 30 19:05:07.040676 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:07.038804    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7bmd\" (UniqueName: \"kubernetes.io/projected/8955fdbb-df1e-4a72-83c2-bef3dbb86a68-kube-api-access-n7bmd\") pod \"payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs\" (UID: \"8955fdbb-df1e-4a72-83c2-bef3dbb86a68\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs"
Oct 30 19:05:07.040676 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:07.038863    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9877e03e-92b6-43c7-83c7-6fdea1d04000-plugins-config-volume\") pod \"payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv\" (UID: \"9877e03e-92b6-43c7-83c7-6fdea1d04000\") " pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv"
Oct 30 19:05:07.040676 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:07.038992    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8955fdbb-df1e-4a72-83c2-bef3dbb86a68-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs\" (UID: \"8955fdbb-df1e-4a72-83c2-bef3dbb86a68\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs"
Oct 30 19:05:07.040676 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:07.039168    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw2bw\" (UniqueName: \"kubernetes.io/projected/9877e03e-92b6-43c7-83c7-6fdea1d04000-kube-api-access-sw2bw\") pod \"payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv\" (UID: \"9877e03e-92b6-43c7-83c7-6fdea1d04000\") " pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv"
Oct 30 19:05:07.041018 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:07.040709    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8955fdbb-df1e-4a72-83c2-bef3dbb86a68-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs\" (UID: \"8955fdbb-df1e-4a72-83c2-bef3dbb86a68\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs"
Oct 30 19:05:07.150114 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:07.139158    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7bmd\" (UniqueName: \"kubernetes.io/projected/8955fdbb-df1e-4a72-83c2-bef3dbb86a68-kube-api-access-n7bmd\") pod \"payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs\" (UID: \"8955fdbb-df1e-4a72-83c2-bef3dbb86a68\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs"
Oct 30 19:05:07.150114 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:07.141345    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw2bw\" (UniqueName: \"kubernetes.io/projected/9877e03e-92b6-43c7-83c7-6fdea1d04000-kube-api-access-sw2bw\") pod \"payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv\" (UID: \"9877e03e-92b6-43c7-83c7-6fdea1d04000\") " pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv"
Oct 30 19:05:07.150114 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:07.141489    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9877e03e-92b6-43c7-83c7-6fdea1d04000-plugins-config-volume\") pod \"payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv\" (UID: \"9877e03e-92b6-43c7-83c7-6fdea1d04000\") " pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv"
Oct 30 19:05:07.150114 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:07.142554    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9877e03e-92b6-43c7-83c7-6fdea1d04000-plugins-config-volume\") pod \"payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv\" (UID: \"9877e03e-92b6-43c7-83c7-6fdea1d04000\") " pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv"
Oct 30 19:05:07.198931 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:07.198886    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw2bw\" (UniqueName: \"kubernetes.io/projected/9877e03e-92b6-43c7-83c7-6fdea1d04000-kube-api-access-sw2bw\") pod \"payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv\" (UID: \"9877e03e-92b6-43c7-83c7-6fdea1d04000\") " pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv"
Oct 30 19:05:07.209906 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:07.209688    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs"
Oct 30 19:05:07.245709 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:07.245626    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv"
Oct 30 19:05:07.381526 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:07.379730    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7" event={"ID":"edd86ca1-466c-4e45-8998-61d7e94fae7d","Type":"ContainerStarted","Data":"4a0e01c8d1b9c81a6fb01bcc321a4ce4c7628aa16eb44f09351a72fe5e4d6dce"}
Oct 30 19:05:07.381526 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:07.379800    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7" event={"ID":"edd86ca1-466c-4e45-8998-61d7e94fae7d","Type":"ContainerStarted","Data":"a98f5d72b4c14a9805084bbccf48e1a5f847887e8c1fb2dd89c4768eee7c7145"}
Oct 30 19:05:07.491551 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:07.491486    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7" podStartSLOduration=2.491471198 podStartE2EDuration="2.491471198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:07.49117799 +0000 UTC m=+1705.017053086" watchObservedRunningTime="2026-10-30 19:05:07.491471198 +0000 UTC m=+1705.017346294"
Oct 30 19:05:07.621469 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:07.621439    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs"]
Oct 30 19:05:07.621878 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:05:07.621853    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8955fdbb_df1e_4a72_83c2_bef3dbb86a68.slice/crio-b5f991ee7349faee14d35991c7e6b84feb6c118e2cb3ab0ebc39531e1bc60b4d WatchSource:0}: Error finding container b5f991ee7349faee14d35991c7e6b84feb6c118e2cb3ab0ebc39531e1bc60b4d: Status 404 returned error can't find the container with id b5f991ee7349faee14d35991c7e6b84feb6c118e2cb3ab0ebc39531e1bc60b4d
Oct 30 19:05:07.669442 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:07.669411    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv"]
Oct 30 19:05:07.672000 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:05:07.671970    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9877e03e_92b6_43c7_83c7_6fdea1d04000.slice/crio-650e368f04292b256a21be8a05ef5aaee9dd93078a92a3950e52c80e187a84f9 WatchSource:0}: Error finding container 650e368f04292b256a21be8a05ef5aaee9dd93078a92a3950e52c80e187a84f9: Status 404 returned error can't find the container with id 650e368f04292b256a21be8a05ef5aaee9dd93078a92a3950e52c80e187a84f9
Oct 30 19:05:08.086625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:08.086046    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:08.090625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:08.088220    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-j76r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:05:08.090625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:08.088277    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7" podUID="edd86ca1-466c-4e45-8998-61d7e94fae7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 19:05:08.395331 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:08.395226    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs" event={"ID":"8955fdbb-df1e-4a72-83c2-bef3dbb86a68","Type":"ContainerStarted","Data":"5a6fac3f1d0c3a0b477d694daeaa82ce2a1d94302adedaf07ac695a0d96d1016"}
Oct 30 19:05:08.395331 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:08.395300    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs"
Oct 30 19:05:08.395331 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:08.395326    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs" event={"ID":"8955fdbb-df1e-4a72-83c2-bef3dbb86a68","Type":"ContainerStarted","Data":"b5f991ee7349faee14d35991c7e6b84feb6c118e2cb3ab0ebc39531e1bc60b4d"}
Oct 30 19:05:08.396945 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:08.396915    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv" event={"ID":"9877e03e-92b6-43c7-83c7-6fdea1d04000","Type":"ContainerStarted","Data":"3dbbe599a4cbde55c2c31d4a0c37792a5fb4dba3670982e9bb7c64cbc050c138"}
Oct 30 19:05:08.397061 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:08.396973    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv" event={"ID":"9877e03e-92b6-43c7-83c7-6fdea1d04000","Type":"ContainerStarted","Data":"650e368f04292b256a21be8a05ef5aaee9dd93078a92a3950e52c80e187a84f9"}
Oct 30 19:05:08.506475 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:08.506423    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs" podStartSLOduration=2.506409081 podStartE2EDuration="2.506409081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:08.504875615 +0000 UTC m=+1706.030750723" watchObservedRunningTime="2026-10-30 19:05:08.506409081 +0000 UTC m=+1706.032284210"
Oct 30 19:05:08.618722 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:08.618657    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv" podStartSLOduration=2.618636809 podStartE2EDuration="2.618636809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:08.615972449 +0000 UTC m=+1706.141847556" watchObservedRunningTime="2026-10-30 19:05:08.618636809 +0000 UTC m=+1706.144511906"
Oct 30 19:05:09.084628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:09.078677    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-j76r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:05:09.084628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:09.078743    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7" podUID="edd86ca1-466c-4e45-8998-61d7e94fae7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 19:05:09.402974 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:09.402881    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv"
Oct 30 19:05:10.085625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:10.077491    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-j76r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:05:10.085625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:10.077554    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7" podUID="edd86ca1-466c-4e45-8998-61d7e94fae7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 19:05:11.087120 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:11.087083    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-j76r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:05:11.087743 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:11.087716    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7" podUID="edd86ca1-466c-4e45-8998-61d7e94fae7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 19:05:11.589359 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:11.589321    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"]
Oct 30 19:05:11.589566 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:11.589532    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8" podUID="47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" containerName="istio-proxy" containerID="cri-o://6e83192df4103c4acb4a3899f2a4fed65fad77013d35a74c4e7d440c31727ed7" gracePeriod=30
Oct 30 19:05:11.983694 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:11.983563    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:05:11.983694 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:11.983636    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8" podUID="47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:05:12.081628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:12.078128    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-j76r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:05:12.081628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:12.078234    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7" podUID="edd86ca1-466c-4e45-8998-61d7e94fae7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 19:05:13.079627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:13.077657    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-j76r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:05:13.079627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:13.077749    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7" podUID="edd86ca1-466c-4e45-8998-61d7e94fae7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 19:05:14.074863 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:14.074828    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-j76r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:05:14.075042 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:14.074886    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7" podUID="edd86ca1-466c-4e45-8998-61d7e94fae7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 19:05:15.861788 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:15.861745    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:15.862283 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:15.862198    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:15.863142 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:15.863114    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:05:16.906047 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:16.906017    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:05:17.017731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.016894    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-kube-api-access-zskxs\" (UniqueName: \"kubernetes.io/projected/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-kube-api-access-zskxs\") pod \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") "
Oct 30 19:05:17.017731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.017038    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-podinfo\") pod \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") "
Oct 30 19:05:17.017731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.017084    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-workload-certs\") pod \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") "
Oct 30 19:05:17.017731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.017168    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-envoy\") pod \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") "
Oct 30 19:05:17.017731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.017209    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-data\") pod \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") "
Oct 30 19:05:17.017731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.017256    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-token\" (UniqueName: \"kubernetes.io/projected/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-token\") pod \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") "
Oct 30 19:05:17.017731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.017412    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-workload-socket\") pod \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") "
Oct 30 19:05:17.017731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.017444    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-workload-certs" pod "47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" (UID: "47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:17.017731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.017468    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istiod-ca-cert\") pod \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") "
Oct 30 19:05:17.018353 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.017783    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-workload-socket" pod "47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" (UID: "47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:17.018353 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.017918    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-data" pod "47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" (UID: "47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:17.018353 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.017959    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-credential-socket\") pod \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\" (UID: \"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd\") "
Oct 30 19:05:17.018717 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.018679    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istiod-ca-cert" pod "47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" (UID: "47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:05:17.018823 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.018778    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:17.018823 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.018799    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:17.018823 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.018812    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:17.019081 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.018970    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-credential-socket" pod "47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" (UID: "47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:17.021179 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.021153    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-podinfo" pod "47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" (UID: "47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:05:17.021300 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.021279    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-token" pod "47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" (UID: "47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:17.021519 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.021490    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-kube-api-access-zskxs" pod "47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" (UID: "47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd"). InnerVolumeSpecName "kube-api-access-zskxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:17.023218 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.023191    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-envoy" pod "47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" (UID: "47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:17.120683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.120485    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zskxs\" (UniqueName: \"kubernetes.io/projected/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-kube-api-access-zskxs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:17.120683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.120521    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:17.120683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.120537    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:17.120683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.120550    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:17.120683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.120564    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:17.120683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.120580    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:17.474233 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.474143    4544 generic.go:356] "Generic (PLEG): container finished" podID="47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" containerID="6e83192df4103c4acb4a3899f2a4fed65fad77013d35a74c4e7d440c31727ed7" exitCode=0
Oct 30 19:05:17.474515 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.474214    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"
Oct 30 19:05:17.474515 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.474248    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8" event={"ID":"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd","Type":"ContainerDied","Data":"6e83192df4103c4acb4a3899f2a4fed65fad77013d35a74c4e7d440c31727ed7"}
Oct 30 19:05:17.474755 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.474566    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8" event={"ID":"47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd","Type":"ContainerDied","Data":"d669ec4cc694ee00970dea1000e63de9ecda3ee871db42211cacfa86ab2c4475"}
Oct 30 19:05:17.474755 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.474598    4544 scope.go:122] "RemoveContainer" containerID="6e83192df4103c4acb4a3899f2a4fed65fad77013d35a74c4e7d440c31727ed7"
Oct 30 19:05:17.491537 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.491482    4544 scope.go:122] "RemoveContainer" containerID="6e83192df4103c4acb4a3899f2a4fed65fad77013d35a74c4e7d440c31727ed7"
Oct 30 19:05:17.491864 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:05:17.491835    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e83192df4103c4acb4a3899f2a4fed65fad77013d35a74c4e7d440c31727ed7\": container with ID starting with 6e83192df4103c4acb4a3899f2a4fed65fad77013d35a74c4e7d440c31727ed7 not found: ID does not exist" containerID="6e83192df4103c4acb4a3899f2a4fed65fad77013d35a74c4e7d440c31727ed7"
Oct 30 19:05:17.492024 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.491995    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e83192df4103c4acb4a3899f2a4fed65fad77013d35a74c4e7d440c31727ed7"} err="failed to get container status \"6e83192df4103c4acb4a3899f2a4fed65fad77013d35a74c4e7d440c31727ed7\": rpc error: code = NotFound desc = could not find container \"6e83192df4103c4acb4a3899f2a4fed65fad77013d35a74c4e7d440c31727ed7\": container with ID starting with 6e83192df4103c4acb4a3899f2a4fed65fad77013d35a74c4e7d440c31727ed7 not found: ID does not exist"
Oct 30 19:05:17.631645 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.631568    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"]
Oct 30 19:05:17.680545 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:17.680453    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-4vhn8"]
Oct 30 19:05:18.616779 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:18.616726    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" path="/var/lib/kubelet/pods/47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd/volumes"
Oct 30 19:05:19.406811 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:19.406781    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs"
Oct 30 19:05:20.422764 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:20.422731    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv"
Oct 30 19:05:23.421375 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.421336    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"]
Oct 30 19:05:23.422142 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.422122    4544 cpu_manager.go:414] "containerMap: removing container" podUID="47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" containerName="istio-proxy"
Oct 30 19:05:23.422196 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.422145    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" containerName="istio-proxy"
Oct 30 19:05:23.422272 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.422261    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="47a6ccf3-4ee4-41f4-aeeb-35e0c996aecd" containerName="istio-proxy"
Oct 30 19:05:23.427425 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.427406    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.477910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.477879    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-091e8f-openshift-default-dockercfg-p2h4z\""
Oct 30 19:05:23.537466 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.537422    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-workload-socket\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.537674 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.537627    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-credential-socket\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.537674 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.537659    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-envoy\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.537807 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.537693    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-data\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.537807 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.537761    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdjss\" (UniqueName: \"kubernetes.io/projected/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-kube-api-access-cdjss\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.537920 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.537806    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-workload-certs\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.537920 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.537857    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-token\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.537920 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.537883    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istiod-ca-cert\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.537920 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.537913    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-podinfo\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.549503 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.549416    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"]
Oct 30 19:05:23.648670 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.644372    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-credential-socket\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.650125 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.650095    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-envoy\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.650324 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.650307    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-data\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.650494 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.650474    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdjss\" (UniqueName: \"kubernetes.io/projected/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-kube-api-access-cdjss\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.650695 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.650677    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-workload-certs\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.650865 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.650849    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-token\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.650989 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.650971    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istiod-ca-cert\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.651148 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.651129    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-podinfo\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.651461 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.651441    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-workload-socket\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.652181 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.649497    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-credential-socket\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.653024 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.653002    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-workload-certs\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.653450 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.653416    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-data\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.655625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.654957    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-workload-socket\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.656778 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.656022    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istiod-ca-cert\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.660196 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.660176    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-envoy\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.662622 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.662582    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-podinfo\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.711984 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.711907    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdjss\" (UniqueName: \"kubernetes.io/projected/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-kube-api-access-cdjss\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.715257 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.715231    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-token\") pod \"e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") " pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:23.744397 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:23.744367    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:24.188676 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:24.188642    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"]
Oct 30 19:05:24.221753 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:05:24.221717    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3c4f1eb_a61c_4184_8ae6_afc8a54e024e.slice/crio-e4b6758a9ca5006027e7bf512b385891eaab903288db8bfa11f1cc2c8e65c076 WatchSource:0}: Error finding container e4b6758a9ca5006027e7bf512b385891eaab903288db8bfa11f1cc2c8e65c076: Status 404 returned error can't find the container with id e4b6758a9ca5006027e7bf512b385891eaab903288db8bfa11f1cc2c8e65c076
Oct 30 19:05:24.224491 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:24.224454    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:05:24.224633 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:24.224537    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:05:24.224633 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:24.224585    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:05:24.618778 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:24.618746    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb" event={"ID":"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e","Type":"ContainerStarted","Data":"89c0a630c5e2f5178593997064c91bf04429f044c020b803cb6ef3bb4c3ae237"}
Oct 30 19:05:24.619181 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:24.618795    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb" event={"ID":"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e","Type":"ContainerStarted","Data":"e4b6758a9ca5006027e7bf512b385891eaab903288db8bfa11f1cc2c8e65c076"}
Oct 30 19:05:24.734637 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:24.734553    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb" podStartSLOduration=1.734537802 podStartE2EDuration="1.734537802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:24.729904842 +0000 UTC m=+1722.255779935" watchObservedRunningTime="2026-10-30 19:05:24.734537802 +0000 UTC m=+1722.260412899"
Oct 30 19:05:24.745225 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:24.745198    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:25.750493 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:25.750464    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:26.630354 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:26.630314    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:26.631317 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:26.631297    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:05:32.696045 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:32.696005    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9"]
Oct 30 19:05:32.731701 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:32.730163    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9"
Oct 30 19:05:32.753566 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:32.753530    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9"]
Oct 30 19:05:32.793412 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:32.793372    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-091e8f\""
Oct 30 19:05:32.794059 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:32.794025    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-091e8f-dockercfg-sbmbh\""
Oct 30 19:05:32.832399 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:32.832358    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bd6f01b-a7cb-48b3-be4c-0c386174170d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9\" (UID: \"7bd6f01b-a7cb-48b3-be4c-0c386174170d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9"
Oct 30 19:05:32.832739 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:32.832715    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m58ds\" (UniqueName: \"kubernetes.io/projected/7bd6f01b-a7cb-48b3-be4c-0c386174170d-kube-api-access-m58ds\") pod \"payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9\" (UID: \"7bd6f01b-a7cb-48b3-be4c-0c386174170d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9"
Oct 30 19:05:32.933962 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:32.933929    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bd6f01b-a7cb-48b3-be4c-0c386174170d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9\" (UID: \"7bd6f01b-a7cb-48b3-be4c-0c386174170d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9"
Oct 30 19:05:32.934177 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:32.934069    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m58ds\" (UniqueName: \"kubernetes.io/projected/7bd6f01b-a7cb-48b3-be4c-0c386174170d-kube-api-access-m58ds\") pod \"payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9\" (UID: \"7bd6f01b-a7cb-48b3-be4c-0c386174170d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9"
Oct 30 19:05:32.934754 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:32.934728    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bd6f01b-a7cb-48b3-be4c-0c386174170d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9\" (UID: \"7bd6f01b-a7cb-48b3-be4c-0c386174170d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9"
Oct 30 19:05:32.989414 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:32.989350    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m58ds\" (UniqueName: \"kubernetes.io/projected/7bd6f01b-a7cb-48b3-be4c-0c386174170d-kube-api-access-m58ds\") pod \"payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9\" (UID: \"7bd6f01b-a7cb-48b3-be4c-0c386174170d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9"
Oct 30 19:05:33.060934 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:33.060909    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9"
Oct 30 19:05:33.443617 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:33.443581    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9"]
Oct 30 19:05:33.444880 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:05:33.444855    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bd6f01b_a7cb_48b3_be4c_0c386174170d.slice/crio-b4d88a305746905d25a4cac8672483306bc88ec2de297d59c3d83372e23b355a WatchSource:0}: Error finding container b4d88a305746905d25a4cac8672483306bc88ec2de297d59c3d83372e23b355a: Status 404 returned error can't find the container with id b4d88a305746905d25a4cac8672483306bc88ec2de297d59c3d83372e23b355a
Oct 30 19:05:33.686226 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:33.686191    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9" event={"ID":"7bd6f01b-a7cb-48b3-be4c-0c386174170d","Type":"ContainerStarted","Data":"e7f2ff83b2c4e24613849fb4ee29138569e8b4287ce7d282b5ee1b16e68ea396"}
Oct 30 19:05:33.686383 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:33.686240    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9" event={"ID":"7bd6f01b-a7cb-48b3-be4c-0c386174170d","Type":"ContainerStarted","Data":"b4d88a305746905d25a4cac8672483306bc88ec2de297d59c3d83372e23b355a"}
Oct 30 19:05:33.686466 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:33.686438    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9"
Oct 30 19:05:33.795963 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:33.795911    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9" podStartSLOduration=1.7958970380000001 podStartE2EDuration="1.795897038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:33.794788143 +0000 UTC m=+1731.320663232" watchObservedRunningTime="2026-10-30 19:05:33.795897038 +0000 UTC m=+1731.321772165"
Oct 30 19:05:35.531744 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:05:35.531705    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:05:41.844815 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:41.844783    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"]
Oct 30 19:05:41.845219 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:41.844978    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l" podUID="76337bdf-89a0-4e5d-9cb0-437962f9dc8c" containerName="istio-proxy" containerID="cri-o://6e401091e3a31d2b6ac2a5ce0d72426b2ee7e9d849c3a991dbcd5dff87300818" gracePeriod=30
Oct 30 19:05:42.061734 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:42.061696    4544 patch_prober.go:29] interesting pod/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:05:42.061925 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:42.061763    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l" podUID="76337bdf-89a0-4e5d-9cb0-437962f9dc8c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:05:44.603205 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.603170    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"]
Oct 30 19:05:44.609077 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.609058    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.660980 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.660950    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w6-0f3f2f-openshift-default-dockercfg-vwv9w\""
Oct 30 19:05:44.696822 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.696760    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9"
Oct 30 19:05:44.710511 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.710481    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"]
Oct 30 19:05:44.712979 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.712949    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-podinfo\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.713150 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.713057    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-data\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.713150 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.713109    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-workload-certs\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.713268 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.713152    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-envoy\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.713330 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.713272    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-token\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.713330 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.713308    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istiod-ca-cert\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.713428 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.713341    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p2gx\" (UniqueName: \"kubernetes.io/projected/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-kube-api-access-8p2gx\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.713428 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.713401    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-credential-socket\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.713581 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.713441    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-workload-socket\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.814411 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.814379    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-token\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.814411 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.814417    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istiod-ca-cert\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.814683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.814438    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p2gx\" (UniqueName: \"kubernetes.io/projected/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-kube-api-access-8p2gx\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.814683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.814481    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-credential-socket\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.814683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.814521    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-workload-socket\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.814683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.814676    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-podinfo\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.814898 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.814762    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-data\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.814898 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.814807    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-workload-certs\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.814898 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.814854    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-envoy\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.815059 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.814954    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-workload-socket\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.815059 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.815013    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-credential-socket\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.815234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.815209    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istiod-ca-cert\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.815303 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.815263    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-data\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.815348 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.815304    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-workload-certs\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.817225 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.817199    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-envoy\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.817551 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.817535    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-podinfo\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.875579 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.875494    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-token\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.882860 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.879930    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p2gx\" (UniqueName: \"kubernetes.io/projected/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-kube-api-access-8p2gx\") pod \"e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") " pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:44.928134 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:44.928105    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:45.374515 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:05:45.374483    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2e66881_cd8a_4bf9_9f15_0ced9d94a3db.slice/crio-93667ee2b5788428c01cd697c8b7d4ca1e46bbdbaea53c0497ae9b7b38c06a39 WatchSource:0}: Error finding container 93667ee2b5788428c01cd697c8b7d4ca1e46bbdbaea53c0497ae9b7b38c06a39: Status 404 returned error can't find the container with id 93667ee2b5788428c01cd697c8b7d4ca1e46bbdbaea53c0497ae9b7b38c06a39
Oct 30 19:05:45.376776 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:45.376749    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"]
Oct 30 19:05:45.377300 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:45.377273    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:05:45.377347 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:45.377335    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:05:45.377382 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:45.377362    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:05:45.816019 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:45.815976    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb" event={"ID":"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db","Type":"ContainerStarted","Data":"0a71aabe4e79fb2504dd62e441b2e8649655cbade08bd02cea9011ad33f2c824"}
Oct 30 19:05:45.816496 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:45.816053    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb" event={"ID":"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db","Type":"ContainerStarted","Data":"93667ee2b5788428c01cd697c8b7d4ca1e46bbdbaea53c0497ae9b7b38c06a39"}
Oct 30 19:05:45.928410 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:45.928377    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:45.937682 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:45.937630    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb" podStartSLOduration=1.937612458 podStartE2EDuration="1.937612458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:45.935618145 +0000 UTC m=+1743.461493236" watchObservedRunningTime="2026-10-30 19:05:45.937612458 +0000 UTC m=+1743.463487547"
Oct 30 19:05:46.934054 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:46.934025    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:47.137879 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.137849    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:05:47.277426 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.277385    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-envoy\") pod \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") "
Oct 30 19:05:47.277668 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.277591    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-data\") pod \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") "
Oct 30 19:05:47.277768 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.277727    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-credential-socket\") pod \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") "
Oct 30 19:05:47.278151 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.277909    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-data" pod "76337bdf-89a0-4e5d-9cb0-437962f9dc8c" (UID: "76337bdf-89a0-4e5d-9cb0-437962f9dc8c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:47.278151 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.278025    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-credential-socket" pod "76337bdf-89a0-4e5d-9cb0-437962f9dc8c" (UID: "76337bdf-89a0-4e5d-9cb0-437962f9dc8c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:47.278151 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.278075    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-kube-api-access-jggwn\" (UniqueName: \"kubernetes.io/projected/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-kube-api-access-jggwn\") pod \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") "
Oct 30 19:05:47.278372 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.278192    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-workload-socket\") pod \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") "
Oct 30 19:05:47.278372 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.278269    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-workload-certs\") pod \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") "
Oct 30 19:05:47.278583 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.278460    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-workload-socket" pod "76337bdf-89a0-4e5d-9cb0-437962f9dc8c" (UID: "76337bdf-89a0-4e5d-9cb0-437962f9dc8c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:47.278583 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.278553    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-workload-certs" pod "76337bdf-89a0-4e5d-9cb0-437962f9dc8c" (UID: "76337bdf-89a0-4e5d-9cb0-437962f9dc8c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:47.278748 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.278683    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-podinfo\") pod \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") "
Oct 30 19:05:47.278748 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.278738    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-token\" (UniqueName: \"kubernetes.io/projected/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-token\") pod \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") "
Oct 30 19:05:47.278857 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.278829    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istiod-ca-cert\") pod \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\" (UID: \"76337bdf-89a0-4e5d-9cb0-437962f9dc8c\") "
Oct 30 19:05:47.279532 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.279496    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istiod-ca-cert" pod "76337bdf-89a0-4e5d-9cb0-437962f9dc8c" (UID: "76337bdf-89a0-4e5d-9cb0-437962f9dc8c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:05:47.280062 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.280039    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:47.280178 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.280162    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:47.280285 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.280271    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:47.280380 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.280367    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:47.280470 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.280458    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:47.282423 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.282362    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-podinfo" pod "76337bdf-89a0-4e5d-9cb0-437962f9dc8c" (UID: "76337bdf-89a0-4e5d-9cb0-437962f9dc8c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:05:47.282804 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.282762    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-envoy" pod "76337bdf-89a0-4e5d-9cb0-437962f9dc8c" (UID: "76337bdf-89a0-4e5d-9cb0-437962f9dc8c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:47.282912 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.282805    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-kube-api-access-jggwn" pod "76337bdf-89a0-4e5d-9cb0-437962f9dc8c" (UID: "76337bdf-89a0-4e5d-9cb0-437962f9dc8c"). InnerVolumeSpecName "kube-api-access-jggwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:47.282975 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.282959    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-token" pod "76337bdf-89a0-4e5d-9cb0-437962f9dc8c" (UID: "76337bdf-89a0-4e5d-9cb0-437962f9dc8c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:47.382519 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.382424    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jggwn\" (UniqueName: \"kubernetes.io/projected/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-kube-api-access-jggwn\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:47.382519 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.382462    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:47.382519 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.382478    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:47.382519 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.382494    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76337bdf-89a0-4e5d-9cb0-437962f9dc8c-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:47.833760 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.833722    4544 generic.go:356] "Generic (PLEG): container finished" podID="76337bdf-89a0-4e5d-9cb0-437962f9dc8c" containerID="6e401091e3a31d2b6ac2a5ce0d72426b2ee7e9d849c3a991dbcd5dff87300818" exitCode=0
Oct 30 19:05:47.833949 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.833784    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"
Oct 30 19:05:47.833949 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.833801    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l" event={"ID":"76337bdf-89a0-4e5d-9cb0-437962f9dc8c","Type":"ContainerDied","Data":"6e401091e3a31d2b6ac2a5ce0d72426b2ee7e9d849c3a991dbcd5dff87300818"}
Oct 30 19:05:47.833949 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.833862    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l" event={"ID":"76337bdf-89a0-4e5d-9cb0-437962f9dc8c","Type":"ContainerDied","Data":"6f8f38e83924b4f22fa97adce2fc9eff0a56966b51f9542d2dda452488ce3747"}
Oct 30 19:05:47.833949 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.833891    4544 scope.go:122] "RemoveContainer" containerID="6e401091e3a31d2b6ac2a5ce0d72426b2ee7e9d849c3a991dbcd5dff87300818"
Oct 30 19:05:47.834302 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.834246    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:47.835563 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.835541    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:05:47.851662 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.851643    4544 scope.go:122] "RemoveContainer" containerID="6e401091e3a31d2b6ac2a5ce0d72426b2ee7e9d849c3a991dbcd5dff87300818"
Oct 30 19:05:47.851992 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:05:47.851970    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e401091e3a31d2b6ac2a5ce0d72426b2ee7e9d849c3a991dbcd5dff87300818\": container with ID starting with 6e401091e3a31d2b6ac2a5ce0d72426b2ee7e9d849c3a991dbcd5dff87300818 not found: ID does not exist" containerID="6e401091e3a31d2b6ac2a5ce0d72426b2ee7e9d849c3a991dbcd5dff87300818"
Oct 30 19:05:47.852063 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.852001    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e401091e3a31d2b6ac2a5ce0d72426b2ee7e9d849c3a991dbcd5dff87300818"} err="failed to get container status \"6e401091e3a31d2b6ac2a5ce0d72426b2ee7e9d849c3a991dbcd5dff87300818\": rpc error: code = NotFound desc = could not find container \"6e401091e3a31d2b6ac2a5ce0d72426b2ee7e9d849c3a991dbcd5dff87300818\": container with ID starting with 6e401091e3a31d2b6ac2a5ce0d72426b2ee7e9d849c3a991dbcd5dff87300818 not found: ID does not exist"
Oct 30 19:05:47.993836 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:47.993798    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"]
Oct 30 19:05:48.040284 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:48.040253    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-t4p7l"]
Oct 30 19:05:48.605075 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:48.605040    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="76337bdf-89a0-4e5d-9cb0-437962f9dc8c" path="/var/lib/kubelet/pods/76337bdf-89a0-4e5d-9cb0-437962f9dc8c/volumes"
Oct 30 19:05:52.379846 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.379794    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6"]
Oct 30 19:05:52.380774 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.380740    4544 cpu_manager.go:414] "containerMap: removing container" podUID="76337bdf-89a0-4e5d-9cb0-437962f9dc8c" containerName="istio-proxy"
Oct 30 19:05:52.380774 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.380765    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76337bdf-89a0-4e5d-9cb0-437962f9dc8c" containerName="istio-proxy"
Oct 30 19:05:52.380989 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.380969    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="76337bdf-89a0-4e5d-9cb0-437962f9dc8c" containerName="istio-proxy"
Oct 30 19:05:52.387147 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.387112    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6"
Oct 30 19:05:52.488677 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.488645    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w6-0f3f2f\""
Oct 30 19:05:52.488908 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.488813    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w6-0f3f2f-dockercfg-t8dlv\""
Oct 30 19:05:52.494383 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.494353    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6"]
Oct 30 19:05:52.505858 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.505730    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl"]
Oct 30 19:05:52.516517 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.516443    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl"
Oct 30 19:05:52.564894 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.564859    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl"]
Oct 30 19:05:52.575629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.572918    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqt9d\" (UniqueName: \"kubernetes.io/projected/21d57f6e-955d-4b8a-bad4-45c9cfdd80ba-kube-api-access-qqt9d\") pod \"payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6\" (UID: \"21d57f6e-955d-4b8a-bad4-45c9cfdd80ba\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6"
Oct 30 19:05:52.575629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.573026    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d57f6e-955d-4b8a-bad4-45c9cfdd80ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6\" (UID: \"21d57f6e-955d-4b8a-bad4-45c9cfdd80ba\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6"
Oct 30 19:05:52.678514 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.678407    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqt9d\" (UniqueName: \"kubernetes.io/projected/21d57f6e-955d-4b8a-bad4-45c9cfdd80ba-kube-api-access-qqt9d\") pod \"payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6\" (UID: \"21d57f6e-955d-4b8a-bad4-45c9cfdd80ba\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6"
Oct 30 19:05:52.678750 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.678509    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d57f6e-955d-4b8a-bad4-45c9cfdd80ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6\" (UID: \"21d57f6e-955d-4b8a-bad4-45c9cfdd80ba\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6"
Oct 30 19:05:52.678750 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.678545    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctfjz\" (UniqueName: \"kubernetes.io/projected/2f6a351b-2333-4f27-a6d6-957830a556a4-kube-api-access-ctfjz\") pod \"payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl\" (UID: \"2f6a351b-2333-4f27-a6d6-957830a556a4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl"
Oct 30 19:05:52.678880 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.678815    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f6a351b-2333-4f27-a6d6-957830a556a4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl\" (UID: \"2f6a351b-2333-4f27-a6d6-957830a556a4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl"
Oct 30 19:05:52.682333 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.681739    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d57f6e-955d-4b8a-bad4-45c9cfdd80ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6\" (UID: \"21d57f6e-955d-4b8a-bad4-45c9cfdd80ba\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6"
Oct 30 19:05:52.736759 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.736722    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqt9d\" (UniqueName: \"kubernetes.io/projected/21d57f6e-955d-4b8a-bad4-45c9cfdd80ba-kube-api-access-qqt9d\") pod \"payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6\" (UID: \"21d57f6e-955d-4b8a-bad4-45c9cfdd80ba\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6"
Oct 30 19:05:52.783194 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.782776    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctfjz\" (UniqueName: \"kubernetes.io/projected/2f6a351b-2333-4f27-a6d6-957830a556a4-kube-api-access-ctfjz\") pod \"payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl\" (UID: \"2f6a351b-2333-4f27-a6d6-957830a556a4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl"
Oct 30 19:05:52.783194 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.783008    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f6a351b-2333-4f27-a6d6-957830a556a4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl\" (UID: \"2f6a351b-2333-4f27-a6d6-957830a556a4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl"
Oct 30 19:05:52.784423 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.784383    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f6a351b-2333-4f27-a6d6-957830a556a4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl\" (UID: \"2f6a351b-2333-4f27-a6d6-957830a556a4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl"
Oct 30 19:05:52.839549 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:52.839523    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctfjz\" (UniqueName: \"kubernetes.io/projected/2f6a351b-2333-4f27-a6d6-957830a556a4-kube-api-access-ctfjz\") pod \"payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl\" (UID: \"2f6a351b-2333-4f27-a6d6-957830a556a4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl"
Oct 30 19:05:53.020722 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:53.020682    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6"
Oct 30 19:05:53.143751 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:53.143716    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl"
Oct 30 19:05:53.412566 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:53.412535    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6"]
Oct 30 19:05:53.413867 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:05:53.413833    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d57f6e_955d_4b8a_bad4_45c9cfdd80ba.slice/crio-13f06ab68baadd604f381de9ef3460a2ec122886f00eb341e31134ddb27dd145 WatchSource:0}: Error finding container 13f06ab68baadd604f381de9ef3460a2ec122886f00eb341e31134ddb27dd145: Status 404 returned error can't find the container with id 13f06ab68baadd604f381de9ef3460a2ec122886f00eb341e31134ddb27dd145
Oct 30 19:05:53.547093 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:53.547059    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl"]
Oct 30 19:05:53.548146 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:05:53.548115    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f6a351b_2333_4f27_a6d6_957830a556a4.slice/crio-227b8b00265711119969b085879ab47c86f94858357c0dc6c2d254204f99646b WatchSource:0}: Error finding container 227b8b00265711119969b085879ab47c86f94858357c0dc6c2d254204f99646b: Status 404 returned error can't find the container with id 227b8b00265711119969b085879ab47c86f94858357c0dc6c2d254204f99646b
Oct 30 19:05:53.885406 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:53.885362    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6" event={"ID":"21d57f6e-955d-4b8a-bad4-45c9cfdd80ba","Type":"ContainerStarted","Data":"92bd3a3d0a42d353d34ccf30f45b902f98c77220797262e9aba3b175b868a200"}
Oct 30 19:05:53.885580 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:53.885441    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6" event={"ID":"21d57f6e-955d-4b8a-bad4-45c9cfdd80ba","Type":"ContainerStarted","Data":"13f06ab68baadd604f381de9ef3460a2ec122886f00eb341e31134ddb27dd145"}
Oct 30 19:05:53.885715 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:53.885591    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6"
Oct 30 19:05:53.887284 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:53.887258    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl" event={"ID":"2f6a351b-2333-4f27-a6d6-957830a556a4","Type":"ContainerStarted","Data":"1006f9b149d26963b54b81fe218f9fc05fd3106e1c0311a639cd03fcc2356414"}
Oct 30 19:05:53.887416 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:53.887306    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl" event={"ID":"2f6a351b-2333-4f27-a6d6-957830a556a4","Type":"ContainerStarted","Data":"227b8b00265711119969b085879ab47c86f94858357c0dc6c2d254204f99646b"}
Oct 30 19:05:53.887416 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:53.887409    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl"
Oct 30 19:05:53.996304 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:53.996257    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6" podStartSLOduration=1.9962433229999998 podStartE2EDuration="1.996243323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:53.995854358 +0000 UTC m=+1751.521729455" watchObservedRunningTime="2026-10-30 19:05:53.996243323 +0000 UTC m=+1751.522118419"
Oct 30 19:05:54.129503 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:05:54.128513    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl" podStartSLOduration=2.128490798 podStartE2EDuration="2.128490798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:54.109953022 +0000 UTC m=+1751.635828120" watchObservedRunningTime="2026-10-30 19:05:54.128490798 +0000 UTC m=+1751.654365897"
Oct 30 19:06:04.901774 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:04.901727    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6"
Oct 30 19:06:04.905457 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:04.905418    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl"
Oct 30 19:06:05.537423 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:06:05.537382    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:06:06.275424 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.275393    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"]
Oct 30 19:06:06.279869 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.279846    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.386898 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.386865    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"]
Oct 30 19:06:06.424288 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.424254    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.424494 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.424431    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.424494 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.424487    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.424654 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.424548    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ccae868-24e4-456f-ad9a-58c17d714b4e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.424748 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.424726    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.424847 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.424829    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.424901 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.424887    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.424975 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.424958    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bjdd\" (UniqueName: \"kubernetes.io/projected/5ccae868-24e4-456f-ad9a-58c17d714b4e-kube-api-access-4bjdd\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.425063 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.425048    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.526114 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.526033    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.526114 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.526077    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.526114 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.526111    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ccae868-24e4-456f-ad9a-58c17d714b4e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.526342 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.526153    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.526342 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.526232    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.526342 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.526271    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.526342 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.526313    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bjdd\" (UniqueName: \"kubernetes.io/projected/5ccae868-24e4-456f-ad9a-58c17d714b4e-kube-api-access-4bjdd\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.526541 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.526397    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.526541 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.526451    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.526809 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.526778    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.526911 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.526875    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.526953 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.526937    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.527078 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.527057    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ccae868-24e4-456f-ad9a-58c17d714b4e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.527152 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.527092    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.528671 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.528652    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.528765 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.528743    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.586868 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.586844    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.586998 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.586902    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bjdd\" (UniqueName: \"kubernetes.io/projected/5ccae868-24e4-456f-ad9a-58c17d714b4e-kube-api-access-4bjdd\") pod \"maas-default-gateway-openshift-default-75d7c785f-zp4pg\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:06.593872 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:06.593852    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:07.073192 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:07.073162    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"]
Oct 30 19:06:07.073784 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:06:07.073753    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ccae868_24e4_456f_ad9a_58c17d714b4e.slice/crio-78f1d2d34ea34308b5863acdf93d3813442c2c3bf3b17807ec937600eaed115e WatchSource:0}: Error finding container 78f1d2d34ea34308b5863acdf93d3813442c2c3bf3b17807ec937600eaed115e: Status 404 returned error can't find the container with id 78f1d2d34ea34308b5863acdf93d3813442c2c3bf3b17807ec937600eaed115e
Oct 30 19:06:07.076757 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:07.076723    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:06:07.076843 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:07.076790    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:06:07.076843 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:07.076821    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:06:08.004309 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:08.004102    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg" event={"ID":"5ccae868-24e4-456f-ad9a-58c17d714b4e","Type":"ContainerStarted","Data":"bc52851088bdd7899f6d74ead1b274fb18331bea864a977142d27030b1410dbd"}
Oct 30 19:06:08.004826 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:08.004373    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg" event={"ID":"5ccae868-24e4-456f-ad9a-58c17d714b4e","Type":"ContainerStarted","Data":"78f1d2d34ea34308b5863acdf93d3813442c2c3bf3b17807ec937600eaed115e"}
Oct 30 19:06:08.144575 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:08.144516    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg" podStartSLOduration=2.144498113 podStartE2EDuration="2.144498113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:08.142776551 +0000 UTC m=+1765.668651660" watchObservedRunningTime="2026-10-30 19:06:08.144498113 +0000 UTC m=+1765.670373211"
Oct 30 19:06:08.594030 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:08.593987    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:08.595826 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:08.595790    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-zp4pg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:06:08.595958 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:08.595856    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg" podUID="5ccae868-24e4-456f-ad9a-58c17d714b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 19:06:09.620962 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:09.620918    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-zp4pg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:06:09.621560 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:09.621090    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg" podUID="5ccae868-24e4-456f-ad9a-58c17d714b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 19:06:10.621631 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:10.620670    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-zp4pg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:06:10.621631 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:10.620740    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg" podUID="5ccae868-24e4-456f-ad9a-58c17d714b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 19:06:11.594259 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:11.594221    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-zp4pg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:06:11.594440 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:11.594284    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg" podUID="5ccae868-24e4-456f-ad9a-58c17d714b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 19:06:12.050890 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.050853    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"]
Oct 30 19:06:12.051379 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.051094    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv" podUID="71bee142-6883-471f-933d-18f2424a1bac" containerName="istio-proxy" containerID="cri-o://edbf6c2e5bb0b8e6ea666133fc8cc68d9852bdb2ae1aacb41f93293cdca3c13b" gracePeriod=30
Oct 30 19:06:12.100636 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.100536    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"]
Oct 30 19:06:12.101014 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.100942    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2" podUID="64da4d53-629c-4015-aaf9-2e3765f9dc5c" containerName="istio-proxy" containerID="cri-o://86146affac30927d6614b5e1e11760e83e883478fe7fe2b8b404d8386332f0a7" gracePeriod=30
Oct 30 19:06:12.108692 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.108666    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"]
Oct 30 19:06:12.108939 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.108901    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks" podUID="94fb0221-0a90-421d-9504-80067ac621ca" containerName="istio-proxy" containerID="cri-o://10fcff2542e940f4afbf65b26b2e7d2a7df9cda0ca4e04b68c3e6243b8eca32c" gracePeriod=30
Oct 30 19:06:12.141371 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.141336    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"]
Oct 30 19:06:12.141813 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.141783    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt" podUID="8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" containerName="istio-proxy" containerID="cri-o://c8034dc7a58213ddbeb20249d60b0445ce4218f1ed562113b4fd36b85249c4b9" gracePeriod=30
Oct 30 19:06:12.151286 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.151258    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"]
Oct 30 19:06:12.151594 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.151574    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk" podUID="0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" containerName="istio-proxy" containerID="cri-o://491b582a839a2c50de4f7231ed0bb51e306dc3094feb01f0a8d202c019d67d21" gracePeriod=30
Oct 30 19:06:12.188699 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.188655    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"]
Oct 30 19:06:12.188932 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.188904    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk" podUID="13a222a4-d62a-4078-bd12-c03a8996dc87" containerName="istio-proxy" containerID="cri-o://7ef2e7870d56ed94d0e6a364395f6fc30150b3fbc766d9c9ed18df9cd7f1737e" gracePeriod=30
Oct 30 19:06:12.208542 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.208505    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"]
Oct 30 19:06:12.209153 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.209120    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7" podUID="48229a00-0acf-4c2b-b14e-d0151692b2d1" containerName="istio-proxy" containerID="cri-o://2236c1e02356ff9b0ede8ace5d7bac7d22377206c0f33ead222556ae47d55fdb" gracePeriod=30
Oct 30 19:06:12.220670 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.220636    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"]
Oct 30 19:06:12.221295 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.221068    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj" podUID="c22154c8-6939-496a-b422-29a775dcbf57" containerName="istio-proxy" containerID="cri-o://4b0cf9aafbfea3e2d0f7dfa6069b60bc0882fef0561e3c4d6d7ccb0fe88d6193" gracePeriod=30
Oct 30 19:06:12.315459 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.310182    4544 patch_prober.go:29] interesting pod/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:06:12.315459 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.310251    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks" podUID="94fb0221-0a90-421d-9504-80067ac621ca" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:06:12.595244 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.595148    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-zp4pg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:06:12.595244 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:12.595222    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg" podUID="5ccae868-24e4-456f-ad9a-58c17d714b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 19:06:13.595059 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:13.595018    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-zp4pg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:06:13.595577 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:13.595101    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg" podUID="5ccae868-24e4-456f-ad9a-58c17d714b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 19:06:14.595455 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:14.595409    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-zp4pg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:06:14.596134 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:14.596086    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg" podUID="5ccae868-24e4-456f-ad9a-58c17d714b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 30 19:06:15.868508 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:15.868456    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:15.869072 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:15.869046    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:15.869996 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:15.869976    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:06:17.139336 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.139158    4544 generic.go:356] "Generic (PLEG): container finished" podID="71bee142-6883-471f-933d-18f2424a1bac" containerID="edbf6c2e5bb0b8e6ea666133fc8cc68d9852bdb2ae1aacb41f93293cdca3c13b" exitCode=0
Oct 30 19:06:17.139336 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.139239    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv" event={"ID":"71bee142-6883-471f-933d-18f2424a1bac","Type":"ContainerDied","Data":"edbf6c2e5bb0b8e6ea666133fc8cc68d9852bdb2ae1aacb41f93293cdca3c13b"}
Oct 30 19:06:17.546294 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.545853    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:06:17.549568 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.549547    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 19:06:17.553331 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.553305    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:06:17.727029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.724425    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/71bee142-6883-471f-933d-18f2424a1bac-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71bee142-6883-471f-933d-18f2424a1bac-istiod-ca-cert\") pod \"71bee142-6883-471f-933d-18f2424a1bac\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") "
Oct 30 19:06:17.727029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.724492    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-credential-socket\") pod \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") "
Oct 30 19:06:17.727029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.724544    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-workload-socket\") pod \"94fb0221-0a90-421d-9504-80067ac621ca\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") "
Oct 30 19:06:17.727029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.724581    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-workload-socket\") pod \"71bee142-6883-471f-933d-18f2424a1bac\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") "
Oct 30 19:06:17.727029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.724629    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-istio-data\") pod \"71bee142-6883-471f-933d-18f2424a1bac\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") "
Oct 30 19:06:17.727029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.724679    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istiod-ca-cert\") pod \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") "
Oct 30 19:06:17.727029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.724707    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71bee142-6883-471f-933d-18f2424a1bac-istio-token\" (UniqueName: \"kubernetes.io/projected/71bee142-6883-471f-933d-18f2424a1bac-istio-token\") pod \"71bee142-6883-471f-933d-18f2424a1bac\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") "
Oct 30 19:06:17.727029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.724737    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-workload-certs\") pod \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") "
Oct 30 19:06:17.727029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.724765    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94fb0221-0a90-421d-9504-80067ac621ca-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94fb0221-0a90-421d-9504-80067ac621ca-istiod-ca-cert\") pod \"94fb0221-0a90-421d-9504-80067ac621ca\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") "
Oct 30 19:06:17.727029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.724804    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-token\" (UniqueName: \"kubernetes.io/projected/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-token\") pod \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") "
Oct 30 19:06:17.727029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.724831    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-data\") pod \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") "
Oct 30 19:06:17.727029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.724858    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-credential-socket\") pod \"71bee142-6883-471f-933d-18f2424a1bac\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") "
Oct 30 19:06:17.727029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.724885    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-workload-certs\") pod \"71bee142-6883-471f-933d-18f2424a1bac\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") "
Oct 30 19:06:17.727029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.726545    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istiod-ca-cert" pod "64da4d53-629c-4015-aaf9-2e3765f9dc5c" (UID: "64da4d53-629c-4015-aaf9-2e3765f9dc5c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:17.727029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.726992    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71bee142-6883-471f-933d-18f2424a1bac-istiod-ca-cert" pod "71bee142-6883-471f-933d-18f2424a1bac" (UID: "71bee142-6883-471f-933d-18f2424a1bac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:17.727950 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.727870    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-data" pod "64da4d53-629c-4015-aaf9-2e3765f9dc5c" (UID: "64da4d53-629c-4015-aaf9-2e3765f9dc5c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.728193    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-workload-certs" pod "64da4d53-629c-4015-aaf9-2e3765f9dc5c" (UID: "64da4d53-629c-4015-aaf9-2e3765f9dc5c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.728897    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94fb0221-0a90-421d-9504-80067ac621ca-kube-api-access-22256\" (UniqueName: \"kubernetes.io/projected/94fb0221-0a90-421d-9504-80067ac621ca-kube-api-access-22256\") pod \"94fb0221-0a90-421d-9504-80067ac621ca\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") "
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.728954    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-workload-certs\") pod \"94fb0221-0a90-421d-9504-80067ac621ca\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") "
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.728980    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-credential-socket" pod "64da4d53-629c-4015-aaf9-2e3765f9dc5c" (UID: "64da4d53-629c-4015-aaf9-2e3765f9dc5c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.729028    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71bee142-6883-471f-933d-18f2424a1bac-kube-api-access-p255p\" (UniqueName: \"kubernetes.io/projected/71bee142-6883-471f-933d-18f2424a1bac-kube-api-access-p255p\") pod \"71bee142-6883-471f-933d-18f2424a1bac\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") "
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.729031    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94fb0221-0a90-421d-9504-80067ac621ca-istiod-ca-cert" pod "94fb0221-0a90-421d-9504-80067ac621ca" (UID: "94fb0221-0a90-421d-9504-80067ac621ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.729155    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-istio-envoy\") pod \"94fb0221-0a90-421d-9504-80067ac621ca\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") "
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.729259    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-workload-socket" pod "94fb0221-0a90-421d-9504-80067ac621ca" (UID: "94fb0221-0a90-421d-9504-80067ac621ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.729471    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-workload-socket" pod "71bee142-6883-471f-933d-18f2424a1bac" (UID: "71bee142-6883-471f-933d-18f2424a1bac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.729500    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-istio-data" pod "71bee142-6883-471f-933d-18f2424a1bac" (UID: "71bee142-6883-471f-933d-18f2424a1bac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.730342    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94fb0221-0a90-421d-9504-80067ac621ca-istio-token\" (UniqueName: \"kubernetes.io/projected/94fb0221-0a90-421d-9504-80067ac621ca-istio-token\") pod \"94fb0221-0a90-421d-9504-80067ac621ca\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") "
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.730395    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-istio-envoy\") pod \"71bee142-6883-471f-933d-18f2424a1bac\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") "
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.730446    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-istio-data\") pod \"94fb0221-0a90-421d-9504-80067ac621ca\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") "
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.730480    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-podinfo\") pod \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") "
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.730545    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64da4d53-629c-4015-aaf9-2e3765f9dc5c-kube-api-access-7pkbs\" (UniqueName: \"kubernetes.io/projected/64da4d53-629c-4015-aaf9-2e3765f9dc5c-kube-api-access-7pkbs\") pod \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") "
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.730579    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/94fb0221-0a90-421d-9504-80067ac621ca-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94fb0221-0a90-421d-9504-80067ac621ca-istio-podinfo\") pod \"94fb0221-0a90-421d-9504-80067ac621ca\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") "
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.730632    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-credential-socket\") pod \"94fb0221-0a90-421d-9504-80067ac621ca\" (UID: \"94fb0221-0a90-421d-9504-80067ac621ca\") "
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.730703    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-workload-socket\") pod \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") "
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.730793    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-envoy\") pod \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\" (UID: \"64da4d53-629c-4015-aaf9-2e3765f9dc5c\") "
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.730830    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/71bee142-6883-471f-933d-18f2424a1bac-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71bee142-6883-471f-933d-18f2424a1bac-istio-podinfo\") pod \"71bee142-6883-471f-933d-18f2424a1bac\" (UID: \"71bee142-6883-471f-933d-18f2424a1bac\") "
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.731232    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-workload-certs" pod "94fb0221-0a90-421d-9504-80067ac621ca" (UID: "94fb0221-0a90-421d-9504-80067ac621ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.732037    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-credential-socket" pod "71bee142-6883-471f-933d-18f2424a1bac" (UID: "71bee142-6883-471f-933d-18f2424a1bac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.732089    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71bee142-6883-471f-933d-18f2424a1bac-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.732113    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.732128    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.732142    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.732158    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.732172    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.732185    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.732200    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94fb0221-0a90-421d-9504-80067ac621ca-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.732213    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.732227    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.732279    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71bee142-6883-471f-933d-18f2424a1bac-istio-token" pod "71bee142-6883-471f-933d-18f2424a1bac" (UID: "71bee142-6883-471f-933d-18f2424a1bac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.732847    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-token" pod "64da4d53-629c-4015-aaf9-2e3765f9dc5c" (UID: "64da4d53-629c-4015-aaf9-2e3765f9dc5c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.733392    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-workload-socket" pod "64da4d53-629c-4015-aaf9-2e3765f9dc5c" (UID: "64da4d53-629c-4015-aaf9-2e3765f9dc5c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.734251    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71bee142-6883-471f-933d-18f2424a1bac-kube-api-access-p255p" pod "71bee142-6883-471f-933d-18f2424a1bac" (UID: "71bee142-6883-471f-933d-18f2424a1bac"). InnerVolumeSpecName "kube-api-access-p255p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.734742    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-istio-data" pod "94fb0221-0a90-421d-9504-80067ac621ca" (UID: "94fb0221-0a90-421d-9504-80067ac621ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.735264    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94fb0221-0a90-421d-9504-80067ac621ca-kube-api-access-22256" pod "94fb0221-0a90-421d-9504-80067ac621ca" (UID: "94fb0221-0a90-421d-9504-80067ac621ca"). InnerVolumeSpecName "kube-api-access-22256". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.736220    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-envoy" pod "64da4d53-629c-4015-aaf9-2e3765f9dc5c" (UID: "64da4d53-629c-4015-aaf9-2e3765f9dc5c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.738368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.736328    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-workload-certs" pod "71bee142-6883-471f-933d-18f2424a1bac" (UID: "71bee142-6883-471f-933d-18f2424a1bac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.740826 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.738523    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-credential-socket" pod "94fb0221-0a90-421d-9504-80067ac621ca" (UID: "94fb0221-0a90-421d-9504-80067ac621ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.740826 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.738781    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-istio-envoy" pod "71bee142-6883-471f-933d-18f2424a1bac" (UID: "71bee142-6883-471f-933d-18f2424a1bac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.741749 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.741724    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-istio-envoy" pod "94fb0221-0a90-421d-9504-80067ac621ca" (UID: "94fb0221-0a90-421d-9504-80067ac621ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.741749 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.741738    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-podinfo" pod "64da4d53-629c-4015-aaf9-2e3765f9dc5c" (UID: "64da4d53-629c-4015-aaf9-2e3765f9dc5c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:06:17.742428 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.742346    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 19:06:17.743667 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.743634    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64da4d53-629c-4015-aaf9-2e3765f9dc5c-kube-api-access-7pkbs" pod "64da4d53-629c-4015-aaf9-2e3765f9dc5c" (UID: "64da4d53-629c-4015-aaf9-2e3765f9dc5c"). InnerVolumeSpecName "kube-api-access-7pkbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:17.745213 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.745158    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71bee142-6883-471f-933d-18f2424a1bac-istio-podinfo" pod "71bee142-6883-471f-933d-18f2424a1bac" (UID: "71bee142-6883-471f-933d-18f2424a1bac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:06:17.745707 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.745669    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/94fb0221-0a90-421d-9504-80067ac621ca-istio-podinfo" pod "94fb0221-0a90-421d-9504-80067ac621ca" (UID: "94fb0221-0a90-421d-9504-80067ac621ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:06:17.748778 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.748665    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 19:06:17.748778 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.748719    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94fb0221-0a90-421d-9504-80067ac621ca-istio-token" pod "94fb0221-0a90-421d-9504-80067ac621ca" (UID: "94fb0221-0a90-421d-9504-80067ac621ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:17.754538 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.754117    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 19:06:17.756753 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.756723    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 19:06:17.797072 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.797033    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 19:06:17.833438 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.833410    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48229a00-0acf-4c2b-b14e-d0151692b2d1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48229a00-0acf-4c2b-b14e-d0151692b2d1-istiod-ca-cert\") pod \"48229a00-0acf-4c2b-b14e-d0151692b2d1\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") "
Oct 30 19:06:17.833647 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.833502    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-workload-certs\") pod \"48229a00-0acf-4c2b-b14e-d0151692b2d1\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") "
Oct 30 19:06:17.833647 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.833563    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-credential-socket\") pod \"48229a00-0acf-4c2b-b14e-d0151692b2d1\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") "
Oct 30 19:06:17.833647 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.833634    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-workload-socket\") pod \"48229a00-0acf-4c2b-b14e-d0151692b2d1\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") "
Oct 30 19:06:17.833840 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.833682    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-token\" (UniqueName: \"kubernetes.io/projected/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-token\") pod \"48229a00-0acf-4c2b-b14e-d0151692b2d1\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") "
Oct 30 19:06:17.833840 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.833728    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-podinfo\") pod \"48229a00-0acf-4c2b-b14e-d0151692b2d1\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") "
Oct 30 19:06:17.833840 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.833796    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48229a00-0acf-4c2b-b14e-d0151692b2d1-kube-api-access-ssk5s\" (UniqueName: \"kubernetes.io/projected/48229a00-0acf-4c2b-b14e-d0151692b2d1-kube-api-access-ssk5s\") pod \"48229a00-0acf-4c2b-b14e-d0151692b2d1\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") "
Oct 30 19:06:17.833840 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.833831    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-envoy\") pod \"48229a00-0acf-4c2b-b14e-d0151692b2d1\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") "
Oct 30 19:06:17.834042 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.833885    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-data\") pod \"48229a00-0acf-4c2b-b14e-d0151692b2d1\" (UID: \"48229a00-0acf-4c2b-b14e-d0151692b2d1\") "
Oct 30 19:06:17.834548 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834371    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.834548 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834397    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94fb0221-0a90-421d-9504-80067ac621ca-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.834548 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834413    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.834548 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834430    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.834548 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834445    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.834548 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834460    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pkbs\" (UniqueName: \"kubernetes.io/projected/64da4d53-629c-4015-aaf9-2e3765f9dc5c-kube-api-access-7pkbs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.834548 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834478    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94fb0221-0a90-421d-9504-80067ac621ca-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.834548 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834493    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94fb0221-0a90-421d-9504-80067ac621ca-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.834548 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834507    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.834548 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834520    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.834548 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834533    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71bee142-6883-471f-933d-18f2424a1bac-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.834548 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834547    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71bee142-6883-471f-933d-18f2424a1bac-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.835283 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834561    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64da4d53-629c-4015-aaf9-2e3765f9dc5c-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.835283 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834577    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.835283 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834592    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71bee142-6883-471f-933d-18f2424a1bac-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.835283 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834621    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22256\" (UniqueName: \"kubernetes.io/projected/94fb0221-0a90-421d-9504-80067ac621ca-kube-api-access-22256\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.835283 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834635    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p255p\" (UniqueName: \"kubernetes.io/projected/71bee142-6883-471f-933d-18f2424a1bac-kube-api-access-p255p\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.835283 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834371    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-workload-socket" pod "48229a00-0acf-4c2b-b14e-d0151692b2d1" (UID: "48229a00-0acf-4c2b-b14e-d0151692b2d1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.835283 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834670    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48229a00-0acf-4c2b-b14e-d0151692b2d1-istiod-ca-cert" pod "48229a00-0acf-4c2b-b14e-d0151692b2d1" (UID: "48229a00-0acf-4c2b-b14e-d0151692b2d1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:17.835283 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.834857    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-workload-certs" pod "48229a00-0acf-4c2b-b14e-d0151692b2d1" (UID: "48229a00-0acf-4c2b-b14e-d0151692b2d1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.835283 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.835127    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-credential-socket" pod "48229a00-0acf-4c2b-b14e-d0151692b2d1" (UID: "48229a00-0acf-4c2b-b14e-d0151692b2d1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.836491 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.835651    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-data" pod "48229a00-0acf-4c2b-b14e-d0151692b2d1" (UID: "48229a00-0acf-4c2b-b14e-d0151692b2d1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.837196 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.837159    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-token" pod "48229a00-0acf-4c2b-b14e-d0151692b2d1" (UID: "48229a00-0acf-4c2b-b14e-d0151692b2d1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:17.839042 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.839013    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-podinfo" pod "48229a00-0acf-4c2b-b14e-d0151692b2d1" (UID: "48229a00-0acf-4c2b-b14e-d0151692b2d1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:06:17.839383 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.839360    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48229a00-0acf-4c2b-b14e-d0151692b2d1-kube-api-access-ssk5s" pod "48229a00-0acf-4c2b-b14e-d0151692b2d1" (UID: "48229a00-0acf-4c2b-b14e-d0151692b2d1"). InnerVolumeSpecName "kube-api-access-ssk5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:17.840785 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.840579    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-envoy" pod "48229a00-0acf-4c2b-b14e-d0151692b2d1" (UID: "48229a00-0acf-4c2b-b14e-d0151692b2d1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.936151 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936050    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-istio-data\") pod \"c22154c8-6939-496a-b422-29a775dcbf57\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") "
Oct 30 19:06:17.936151 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936122    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-podinfo\") pod \"13a222a4-d62a-4078-bd12-c03a8996dc87\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") "
Oct 30 19:06:17.936377 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936173    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-podinfo\") pod \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") "
Oct 30 19:06:17.936377 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936253    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-kube-api-access-srfn4\" (UniqueName: \"kubernetes.io/projected/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-kube-api-access-srfn4\") pod \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") "
Oct 30 19:06:17.936377 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936283    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-credential-socket\") pod \"c22154c8-6939-496a-b422-29a775dcbf57\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") "
Oct 30 19:06:17.936377 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936316    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-envoy\") pod \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") "
Oct 30 19:06:17.936588 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936377    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istiod-ca-cert\") pod \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") "
Oct 30 19:06:17.936588 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936378    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-istio-data" pod "c22154c8-6939-496a-b422-29a775dcbf57" (UID: "c22154c8-6939-496a-b422-29a775dcbf57"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.936588 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936422    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-data\") pod \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") "
Oct 30 19:06:17.936588 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936448    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-istio-envoy\") pod \"c22154c8-6939-496a-b422-29a775dcbf57\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") "
Oct 30 19:06:17.936588 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936477    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-credential-socket\") pod \"13a222a4-d62a-4078-bd12-c03a8996dc87\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") "
Oct 30 19:06:17.936588 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936512    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-workload-certs\") pod \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") "
Oct 30 19:06:17.936588 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936542    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-envoy\") pod \"13a222a4-d62a-4078-bd12-c03a8996dc87\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") "
Oct 30 19:06:17.936588 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936573    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-workload-socket\") pod \"c22154c8-6939-496a-b422-29a775dcbf57\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") "
Oct 30 19:06:17.937020 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936644    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-kube-api-access-5rlbz\" (UniqueName: \"kubernetes.io/projected/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-kube-api-access-5rlbz\") pod \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") "
Oct 30 19:06:17.937020 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936685    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-token\" (UniqueName: \"kubernetes.io/projected/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-token\") pod \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") "
Oct 30 19:06:17.937020 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936719    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-data\") pod \"13a222a4-d62a-4078-bd12-c03a8996dc87\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") "
Oct 30 19:06:17.937020 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936771    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-workload-certs\") pod \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") "
Oct 30 19:06:17.937020 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936809    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-token\" (UniqueName: \"kubernetes.io/projected/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-token\") pod \"13a222a4-d62a-4078-bd12-c03a8996dc87\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") "
Oct 30 19:06:17.937020 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936856    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-data\") pod \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") "
Oct 30 19:06:17.937020 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936895    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-token\" (UniqueName: \"kubernetes.io/projected/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-token\") pod \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") "
Oct 30 19:06:17.937020 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936934    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-workload-socket\") pod \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") "
Oct 30 19:06:17.937020 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.936965    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-workload-socket\") pod \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") "
Oct 30 19:06:17.937469 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.937054    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-envoy\") pod \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") "
Oct 30 19:06:17.937469 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.937104    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istiod-ca-cert\") pod \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") "
Oct 30 19:06:17.937469 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.937186    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-workload-socket\") pod \"13a222a4-d62a-4078-bd12-c03a8996dc87\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") "
Oct 30 19:06:17.937469 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.937239    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-credential-socket\") pod \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") "
Oct 30 19:06:17.937469 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.937269    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-workload-certs\") pod \"c22154c8-6939-496a-b422-29a775dcbf57\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") "
Oct 30 19:06:17.937469 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.937311    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c22154c8-6939-496a-b422-29a775dcbf57-istio-token\" (UniqueName: \"kubernetes.io/projected/c22154c8-6939-496a-b422-29a775dcbf57-istio-token\") pod \"c22154c8-6939-496a-b422-29a775dcbf57\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") "
Oct 30 19:06:17.937469 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.937343    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c22154c8-6939-496a-b422-29a775dcbf57-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c22154c8-6939-496a-b422-29a775dcbf57-istiod-ca-cert\") pod \"c22154c8-6939-496a-b422-29a775dcbf57\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") "
Oct 30 19:06:17.937469 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.937410    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c22154c8-6939-496a-b422-29a775dcbf57-kube-api-access-dx9n5\" (UniqueName: \"kubernetes.io/projected/c22154c8-6939-496a-b422-29a775dcbf57-kube-api-access-dx9n5\") pod \"c22154c8-6939-496a-b422-29a775dcbf57\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") "
Oct 30 19:06:17.937469 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.937448    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-credential-socket\") pod \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\" (UID: \"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b\") "
Oct 30 19:06:17.937986 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.937486    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/13a222a4-d62a-4078-bd12-c03a8996dc87-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13a222a4-d62a-4078-bd12-c03a8996dc87-istiod-ca-cert\") pod \"13a222a4-d62a-4078-bd12-c03a8996dc87\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") "
Oct 30 19:06:17.937986 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.937500    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-credential-socket" pod "c22154c8-6939-496a-b422-29a775dcbf57" (UID: "c22154c8-6939-496a-b422-29a775dcbf57"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.937986 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.937524    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13a222a4-d62a-4078-bd12-c03a8996dc87-kube-api-access-rc47d\" (UniqueName: \"kubernetes.io/projected/13a222a4-d62a-4078-bd12-c03a8996dc87-kube-api-access-rc47d\") pod \"13a222a4-d62a-4078-bd12-c03a8996dc87\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") "
Oct 30 19:06:17.937986 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.937554    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-workload-certs\") pod \"13a222a4-d62a-4078-bd12-c03a8996dc87\" (UID: \"13a222a4-d62a-4078-bd12-c03a8996dc87\") "
Oct 30 19:06:17.937986 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.937641    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istiod-ca-cert" pod "8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" (UID: "8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:17.937986 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.937729    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-data" pod "0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" (UID: "0da27f8b-2d0d-4737-a02f-ab1e1d7d150b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.937986 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.937597    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c22154c8-6939-496a-b422-29a775dcbf57-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c22154c8-6939-496a-b422-29a775dcbf57-istio-podinfo\") pod \"c22154c8-6939-496a-b422-29a775dcbf57\" (UID: \"c22154c8-6939-496a-b422-29a775dcbf57\") "
Oct 30 19:06:17.937986 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.937805    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-podinfo\") pod \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\" (UID: \"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d\") "
Oct 30 19:06:17.937986 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.937903    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-workload-socket" pod "8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" (UID: "8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.938417 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.938109    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-credential-socket" pod "13a222a4-d62a-4078-bd12-c03a8996dc87" (UID: "13a222a4-d62a-4078-bd12-c03a8996dc87"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.938948 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.938748    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.938948 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.938772    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.938948 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.938780    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-data" pod "13a222a4-d62a-4078-bd12-c03a8996dc87" (UID: "13a222a4-d62a-4078-bd12-c03a8996dc87"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.938948 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.938787    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.938948 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.938825    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48229a00-0acf-4c2b-b14e-d0151692b2d1-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.938948 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.938841    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.938948 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.938856    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.938948 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.938870    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.938948 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.938884    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.938948 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.938898    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.938948 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.938911    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.939581 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.938962    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.939581 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.938979    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssk5s\" (UniqueName: \"kubernetes.io/projected/48229a00-0acf-4c2b-b14e-d0151692b2d1-kube-api-access-ssk5s\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.939581 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.938994    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48229a00-0acf-4c2b-b14e-d0151692b2d1-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.939581 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.939018    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.939581 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.939033    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:17.939581 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.939226    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-workload-certs" pod "0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" (UID: "0da27f8b-2d0d-4737-a02f-ab1e1d7d150b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.940221 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.939961    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-workload-socket" pod "c22154c8-6939-496a-b422-29a775dcbf57" (UID: "c22154c8-6939-496a-b422-29a775dcbf57"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.940356 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.940331    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c22154c8-6939-496a-b422-29a775dcbf57-istiod-ca-cert" pod "c22154c8-6939-496a-b422-29a775dcbf57" (UID: "c22154c8-6939-496a-b422-29a775dcbf57"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:17.940462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.940425    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-credential-socket" pod "8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" (UID: "8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.940680 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.940631    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-workload-certs" pod "c22154c8-6939-496a-b422-29a775dcbf57" (UID: "c22154c8-6939-496a-b422-29a775dcbf57"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.940779 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.940665    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13a222a4-d62a-4078-bd12-c03a8996dc87-istiod-ca-cert" pod "13a222a4-d62a-4078-bd12-c03a8996dc87" (UID: "13a222a4-d62a-4078-bd12-c03a8996dc87"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:17.940843 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.940790    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istiod-ca-cert" pod "0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" (UID: "0da27f8b-2d0d-4737-a02f-ab1e1d7d150b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:17.941096 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.941050    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-workload-socket" pod "13a222a4-d62a-4078-bd12-c03a8996dc87" (UID: "13a222a4-d62a-4078-bd12-c03a8996dc87"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.941184 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.941112    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-workload-certs" pod "8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" (UID: "8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.941813 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.941770    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c22154c8-6939-496a-b422-29a775dcbf57-istio-podinfo" pod "c22154c8-6939-496a-b422-29a775dcbf57" (UID: "c22154c8-6939-496a-b422-29a775dcbf57"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:06:17.942043 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.942018    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-workload-certs" pod "13a222a4-d62a-4078-bd12-c03a8996dc87" (UID: "13a222a4-d62a-4078-bd12-c03a8996dc87"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.942910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.942621    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-data" pod "8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" (UID: "8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.942910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.942859    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-credential-socket" pod "0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" (UID: "0da27f8b-2d0d-4737-a02f-ab1e1d7d150b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.943107 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.942980    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-workload-socket" pod "0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" (UID: "0da27f8b-2d0d-4737-a02f-ab1e1d7d150b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.944532 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.944496    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-token" pod "8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" (UID: "8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:17.945087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.944856    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-envoy" pod "8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" (UID: "8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.945087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.944882    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-podinfo" pod "13a222a4-d62a-4078-bd12-c03a8996dc87" (UID: "13a222a4-d62a-4078-bd12-c03a8996dc87"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:06:17.945087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.944920    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-podinfo" pod "8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" (UID: "8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:06:17.945087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.944942    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-podinfo" pod "0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" (UID: "0da27f8b-2d0d-4737-a02f-ab1e1d7d150b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:06:17.945087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.944992    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-envoy" pod "13a222a4-d62a-4078-bd12-c03a8996dc87" (UID: "13a222a4-d62a-4078-bd12-c03a8996dc87"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.945087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.944999    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-envoy" pod "0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" (UID: "0da27f8b-2d0d-4737-a02f-ab1e1d7d150b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.945087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.944997    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-token" pod "0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" (UID: "0da27f8b-2d0d-4737-a02f-ab1e1d7d150b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:17.945087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.945050    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-istio-envoy" pod "c22154c8-6939-496a-b422-29a775dcbf57" (UID: "c22154c8-6939-496a-b422-29a775dcbf57"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:17.945840 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.945799    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13a222a4-d62a-4078-bd12-c03a8996dc87-kube-api-access-rc47d" pod "13a222a4-d62a-4078-bd12-c03a8996dc87" (UID: "13a222a4-d62a-4078-bd12-c03a8996dc87"). InnerVolumeSpecName "kube-api-access-rc47d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:17.946065 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.946037    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-kube-api-access-srfn4" pod "0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" (UID: "0da27f8b-2d0d-4737-a02f-ab1e1d7d150b"). InnerVolumeSpecName "kube-api-access-srfn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:17.946157 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.946139    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-kube-api-access-5rlbz" pod "8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" (UID: "8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d"). InnerVolumeSpecName "kube-api-access-5rlbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:17.946339 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.946318    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c22154c8-6939-496a-b422-29a775dcbf57-istio-token" pod "c22154c8-6939-496a-b422-29a775dcbf57" (UID: "c22154c8-6939-496a-b422-29a775dcbf57"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:17.946684 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.946663    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-token" pod "13a222a4-d62a-4078-bd12-c03a8996dc87" (UID: "13a222a4-d62a-4078-bd12-c03a8996dc87"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:17.946864 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:17.946847    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c22154c8-6939-496a-b422-29a775dcbf57-kube-api-access-dx9n5" pod "c22154c8-6939-496a-b422-29a775dcbf57" (UID: "c22154c8-6939-496a-b422-29a775dcbf57"). InnerVolumeSpecName "kube-api-access-dx9n5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:18.040479 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040445    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040479 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040473    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040479 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040483    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040494    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040506    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c22154c8-6939-496a-b422-29a775dcbf57-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040515    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c22154c8-6939-496a-b422-29a775dcbf57-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040523    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx9n5\" (UniqueName: \"kubernetes.io/projected/c22154c8-6939-496a-b422-29a775dcbf57-kube-api-access-dx9n5\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040534    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040549    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13a222a4-d62a-4078-bd12-c03a8996dc87-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040561    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rc47d\" (UniqueName: \"kubernetes.io/projected/13a222a4-d62a-4078-bd12-c03a8996dc87-kube-api-access-rc47d\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040572    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040584    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c22154c8-6939-496a-b422-29a775dcbf57-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040592    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040617    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040630    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040644    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srfn4\" (UniqueName: \"kubernetes.io/projected/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-kube-api-access-srfn4\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040652    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040662    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040671    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040683    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040694    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22154c8-6939-496a-b422-29a775dcbf57-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040703    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rlbz\" (UniqueName: \"kubernetes.io/projected/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-kube-api-access-5rlbz\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040712    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040721    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040729    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040737    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13a222a4-d62a-4078-bd12-c03a8996dc87-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040744    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040753    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040761    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.040827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.040769    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:18.151474 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.151434    4544 generic.go:356] "Generic (PLEG): container finished" podID="48229a00-0acf-4c2b-b14e-d0151692b2d1" containerID="2236c1e02356ff9b0ede8ace5d7bac7d22377206c0f33ead222556ae47d55fdb" exitCode=0
Oct 30 19:06:18.151989 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.151497    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"
Oct 30 19:06:18.151989 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.151537    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7" event={"ID":"48229a00-0acf-4c2b-b14e-d0151692b2d1","Type":"ContainerDied","Data":"2236c1e02356ff9b0ede8ace5d7bac7d22377206c0f33ead222556ae47d55fdb"}
Oct 30 19:06:18.151989 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.151591    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7" event={"ID":"48229a00-0acf-4c2b-b14e-d0151692b2d1","Type":"ContainerDied","Data":"18ad5fa11d98d419cf3a59e8dade358ac41a864ccf7d99357b9c1ea3a4d049d5"}
Oct 30 19:06:18.151989 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.151656    4544 scope.go:122] "RemoveContainer" containerID="2236c1e02356ff9b0ede8ace5d7bac7d22377206c0f33ead222556ae47d55fdb"
Oct 30 19:06:18.159937 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.159904    4544 generic.go:356] "Generic (PLEG): container finished" podID="c22154c8-6939-496a-b422-29a775dcbf57" containerID="4b0cf9aafbfea3e2d0f7dfa6069b60bc0882fef0561e3c4d6d7ccb0fe88d6193" exitCode=0
Oct 30 19:06:18.160085 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.160044    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"
Oct 30 19:06:18.172718 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.163699    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj" event={"ID":"c22154c8-6939-496a-b422-29a775dcbf57","Type":"ContainerDied","Data":"4b0cf9aafbfea3e2d0f7dfa6069b60bc0882fef0561e3c4d6d7ccb0fe88d6193"}
Oct 30 19:06:18.172718 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.163765    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj" event={"ID":"c22154c8-6939-496a-b422-29a775dcbf57","Type":"ContainerDied","Data":"41c8962fb7b7654be91160937c8f202e57d57507b2be25df3e34c75c50308cc9"}
Oct 30 19:06:18.172718 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.165454    4544 generic.go:356] "Generic (PLEG): container finished" podID="8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" containerID="c8034dc7a58213ddbeb20249d60b0445ce4218f1ed562113b4fd36b85249c4b9" exitCode=0
Oct 30 19:06:18.172718 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.165523    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt" event={"ID":"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d","Type":"ContainerDied","Data":"c8034dc7a58213ddbeb20249d60b0445ce4218f1ed562113b4fd36b85249c4b9"}
Oct 30 19:06:18.172718 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.165556    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt" event={"ID":"8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d","Type":"ContainerDied","Data":"20c52d66c60d3752a88efc2d44e358ac862d7a041138ef897f346265ffaed654"}
Oct 30 19:06:18.172718 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.165632    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"
Oct 30 19:06:18.182788 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.175173    4544 scope.go:122] "RemoveContainer" containerID="2236c1e02356ff9b0ede8ace5d7bac7d22377206c0f33ead222556ae47d55fdb"
Oct 30 19:06:18.182788 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.175832    4544 generic.go:356] "Generic (PLEG): container finished" podID="13a222a4-d62a-4078-bd12-c03a8996dc87" containerID="7ef2e7870d56ed94d0e6a364395f6fc30150b3fbc766d9c9ed18df9cd7f1737e" exitCode=0
Oct 30 19:06:18.182788 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.175951    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk" event={"ID":"13a222a4-d62a-4078-bd12-c03a8996dc87","Type":"ContainerDied","Data":"7ef2e7870d56ed94d0e6a364395f6fc30150b3fbc766d9c9ed18df9cd7f1737e"}
Oct 30 19:06:18.182788 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.175991    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk" event={"ID":"13a222a4-d62a-4078-bd12-c03a8996dc87","Type":"ContainerDied","Data":"fceffd05db4b38608f2788e21d0fce4fbacbbfcc729ec03044d3834f885983f7"}
Oct 30 19:06:18.182788 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:06:18.176045    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2236c1e02356ff9b0ede8ace5d7bac7d22377206c0f33ead222556ae47d55fdb\": container with ID starting with 2236c1e02356ff9b0ede8ace5d7bac7d22377206c0f33ead222556ae47d55fdb not found: ID does not exist" containerID="2236c1e02356ff9b0ede8ace5d7bac7d22377206c0f33ead222556ae47d55fdb"
Oct 30 19:06:18.182788 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.176071    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2236c1e02356ff9b0ede8ace5d7bac7d22377206c0f33ead222556ae47d55fdb"} err="failed to get container status \"2236c1e02356ff9b0ede8ace5d7bac7d22377206c0f33ead222556ae47d55fdb\": rpc error: code = NotFound desc = could not find container \"2236c1e02356ff9b0ede8ace5d7bac7d22377206c0f33ead222556ae47d55fdb\": container with ID starting with 2236c1e02356ff9b0ede8ace5d7bac7d22377206c0f33ead222556ae47d55fdb not found: ID does not exist"
Oct 30 19:06:18.182788 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.176091    4544 scope.go:122] "RemoveContainer" containerID="4b0cf9aafbfea3e2d0f7dfa6069b60bc0882fef0561e3c4d6d7ccb0fe88d6193"
Oct 30 19:06:18.182788 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.176184    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"
Oct 30 19:06:18.184753 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.183561    4544 generic.go:356] "Generic (PLEG): container finished" podID="64da4d53-629c-4015-aaf9-2e3765f9dc5c" containerID="86146affac30927d6614b5e1e11760e83e883478fe7fe2b8b404d8386332f0a7" exitCode=0
Oct 30 19:06:18.184753 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.183628    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"
Oct 30 19:06:18.184753 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.183635    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2" event={"ID":"64da4d53-629c-4015-aaf9-2e3765f9dc5c","Type":"ContainerDied","Data":"86146affac30927d6614b5e1e11760e83e883478fe7fe2b8b404d8386332f0a7"}
Oct 30 19:06:18.185342 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.185322    4544 generic.go:356] "Generic (PLEG): container finished" podID="94fb0221-0a90-421d-9504-80067ac621ca" containerID="10fcff2542e940f4afbf65b26b2e7d2a7df9cda0ca4e04b68c3e6243b8eca32c" exitCode=0
Oct 30 19:06:18.187589 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.185530    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"
Oct 30 19:06:18.188894 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.188867    4544 generic.go:356] "Generic (PLEG): container finished" podID="0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" containerID="491b582a839a2c50de4f7231ed0bb51e306dc3094feb01f0a8d202c019d67d21" exitCode=0
Oct 30 19:06:18.189014 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.189003    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"
Oct 30 19:06:18.194197 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.192339    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"
Oct 30 19:06:18.194197 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.192944    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2" event={"ID":"64da4d53-629c-4015-aaf9-2e3765f9dc5c","Type":"ContainerDied","Data":"a655b18602c45c3a45e6077b5c5d0cec49277aec8ddfd97a910f749226f03e30"}
Oct 30 19:06:18.194197 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.192987    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks" event={"ID":"94fb0221-0a90-421d-9504-80067ac621ca","Type":"ContainerDied","Data":"10fcff2542e940f4afbf65b26b2e7d2a7df9cda0ca4e04b68c3e6243b8eca32c"}
Oct 30 19:06:18.194197 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.193014    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks" event={"ID":"94fb0221-0a90-421d-9504-80067ac621ca","Type":"ContainerDied","Data":"c71d65a7a9913b8c028f8cd22460207da43735ac8f240688df8f8fee2f13586c"}
Oct 30 19:06:18.194197 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.193035    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk" event={"ID":"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b","Type":"ContainerDied","Data":"491b582a839a2c50de4f7231ed0bb51e306dc3094feb01f0a8d202c019d67d21"}
Oct 30 19:06:18.194197 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.193060    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk" event={"ID":"0da27f8b-2d0d-4737-a02f-ab1e1d7d150b","Type":"ContainerDied","Data":"597bdea135667d6db6b2af41eadebab44504ad991fe7aae2978cd02dbcc9a0d1"}
Oct 30 19:06:18.194197 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.193081    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv" event={"ID":"71bee142-6883-471f-933d-18f2424a1bac","Type":"ContainerDied","Data":"d1c16ac4c8fb89399f1256b20fe8938e1924321ad073d2e008986a26d58c3ecd"}
Oct 30 19:06:18.203876 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.203850    4544 scope.go:122] "RemoveContainer" containerID="4b0cf9aafbfea3e2d0f7dfa6069b60bc0882fef0561e3c4d6d7ccb0fe88d6193"
Oct 30 19:06:18.204876 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:06:18.204850    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b0cf9aafbfea3e2d0f7dfa6069b60bc0882fef0561e3c4d6d7ccb0fe88d6193\": container with ID starting with 4b0cf9aafbfea3e2d0f7dfa6069b60bc0882fef0561e3c4d6d7ccb0fe88d6193 not found: ID does not exist" containerID="4b0cf9aafbfea3e2d0f7dfa6069b60bc0882fef0561e3c4d6d7ccb0fe88d6193"
Oct 30 19:06:18.205022 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.204887    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b0cf9aafbfea3e2d0f7dfa6069b60bc0882fef0561e3c4d6d7ccb0fe88d6193"} err="failed to get container status \"4b0cf9aafbfea3e2d0f7dfa6069b60bc0882fef0561e3c4d6d7ccb0fe88d6193\": rpc error: code = NotFound desc = could not find container \"4b0cf9aafbfea3e2d0f7dfa6069b60bc0882fef0561e3c4d6d7ccb0fe88d6193\": container with ID starting with 4b0cf9aafbfea3e2d0f7dfa6069b60bc0882fef0561e3c4d6d7ccb0fe88d6193 not found: ID does not exist"
Oct 30 19:06:18.205087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.205025    4544 scope.go:122] "RemoveContainer" containerID="c8034dc7a58213ddbeb20249d60b0445ce4218f1ed562113b4fd36b85249c4b9"
Oct 30 19:06:18.233292 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.233264    4544 scope.go:122] "RemoveContainer" containerID="c8034dc7a58213ddbeb20249d60b0445ce4218f1ed562113b4fd36b85249c4b9"
Oct 30 19:06:18.233612 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:06:18.233581    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8034dc7a58213ddbeb20249d60b0445ce4218f1ed562113b4fd36b85249c4b9\": container with ID starting with c8034dc7a58213ddbeb20249d60b0445ce4218f1ed562113b4fd36b85249c4b9 not found: ID does not exist" containerID="c8034dc7a58213ddbeb20249d60b0445ce4218f1ed562113b4fd36b85249c4b9"
Oct 30 19:06:18.233682 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.233624    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8034dc7a58213ddbeb20249d60b0445ce4218f1ed562113b4fd36b85249c4b9"} err="failed to get container status \"c8034dc7a58213ddbeb20249d60b0445ce4218f1ed562113b4fd36b85249c4b9\": rpc error: code = NotFound desc = could not find container \"c8034dc7a58213ddbeb20249d60b0445ce4218f1ed562113b4fd36b85249c4b9\": container with ID starting with c8034dc7a58213ddbeb20249d60b0445ce4218f1ed562113b4fd36b85249c4b9 not found: ID does not exist"
Oct 30 19:06:18.233682 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.233649    4544 scope.go:122] "RemoveContainer" containerID="7ef2e7870d56ed94d0e6a364395f6fc30150b3fbc766d9c9ed18df9cd7f1737e"
Oct 30 19:06:18.243756 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.243740    4544 scope.go:122] "RemoveContainer" containerID="7ef2e7870d56ed94d0e6a364395f6fc30150b3fbc766d9c9ed18df9cd7f1737e"
Oct 30 19:06:18.244050 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:06:18.244023    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ef2e7870d56ed94d0e6a364395f6fc30150b3fbc766d9c9ed18df9cd7f1737e\": container with ID starting with 7ef2e7870d56ed94d0e6a364395f6fc30150b3fbc766d9c9ed18df9cd7f1737e not found: ID does not exist" containerID="7ef2e7870d56ed94d0e6a364395f6fc30150b3fbc766d9c9ed18df9cd7f1737e"
Oct 30 19:06:18.244098 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.244061    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ef2e7870d56ed94d0e6a364395f6fc30150b3fbc766d9c9ed18df9cd7f1737e"} err="failed to get container status \"7ef2e7870d56ed94d0e6a364395f6fc30150b3fbc766d9c9ed18df9cd7f1737e\": rpc error: code = NotFound desc = could not find container \"7ef2e7870d56ed94d0e6a364395f6fc30150b3fbc766d9c9ed18df9cd7f1737e\": container with ID starting with 7ef2e7870d56ed94d0e6a364395f6fc30150b3fbc766d9c9ed18df9cd7f1737e not found: ID does not exist"
Oct 30 19:06:18.244098 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.244082    4544 scope.go:122] "RemoveContainer" containerID="86146affac30927d6614b5e1e11760e83e883478fe7fe2b8b404d8386332f0a7"
Oct 30 19:06:18.252716 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.252697    4544 scope.go:122] "RemoveContainer" containerID="86146affac30927d6614b5e1e11760e83e883478fe7fe2b8b404d8386332f0a7"
Oct 30 19:06:18.252981 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:06:18.252960    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86146affac30927d6614b5e1e11760e83e883478fe7fe2b8b404d8386332f0a7\": container with ID starting with 86146affac30927d6614b5e1e11760e83e883478fe7fe2b8b404d8386332f0a7 not found: ID does not exist" containerID="86146affac30927d6614b5e1e11760e83e883478fe7fe2b8b404d8386332f0a7"
Oct 30 19:06:18.253031 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.252989    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86146affac30927d6614b5e1e11760e83e883478fe7fe2b8b404d8386332f0a7"} err="failed to get container status \"86146affac30927d6614b5e1e11760e83e883478fe7fe2b8b404d8386332f0a7\": rpc error: code = NotFound desc = could not find container \"86146affac30927d6614b5e1e11760e83e883478fe7fe2b8b404d8386332f0a7\": container with ID starting with 86146affac30927d6614b5e1e11760e83e883478fe7fe2b8b404d8386332f0a7 not found: ID does not exist"
Oct 30 19:06:18.253031 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.253005    4544 scope.go:122] "RemoveContainer" containerID="10fcff2542e940f4afbf65b26b2e7d2a7df9cda0ca4e04b68c3e6243b8eca32c"
Oct 30 19:06:18.261741 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.261711    4544 scope.go:122] "RemoveContainer" containerID="10fcff2542e940f4afbf65b26b2e7d2a7df9cda0ca4e04b68c3e6243b8eca32c"
Oct 30 19:06:18.261982 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:06:18.261965    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10fcff2542e940f4afbf65b26b2e7d2a7df9cda0ca4e04b68c3e6243b8eca32c\": container with ID starting with 10fcff2542e940f4afbf65b26b2e7d2a7df9cda0ca4e04b68c3e6243b8eca32c not found: ID does not exist" containerID="10fcff2542e940f4afbf65b26b2e7d2a7df9cda0ca4e04b68c3e6243b8eca32c"
Oct 30 19:06:18.262029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.261987    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10fcff2542e940f4afbf65b26b2e7d2a7df9cda0ca4e04b68c3e6243b8eca32c"} err="failed to get container status \"10fcff2542e940f4afbf65b26b2e7d2a7df9cda0ca4e04b68c3e6243b8eca32c\": rpc error: code = NotFound desc = could not find container \"10fcff2542e940f4afbf65b26b2e7d2a7df9cda0ca4e04b68c3e6243b8eca32c\": container with ID starting with 10fcff2542e940f4afbf65b26b2e7d2a7df9cda0ca4e04b68c3e6243b8eca32c not found: ID does not exist"
Oct 30 19:06:18.262029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.262002    4544 scope.go:122] "RemoveContainer" containerID="491b582a839a2c50de4f7231ed0bb51e306dc3094feb01f0a8d202c019d67d21"
Oct 30 19:06:18.279190 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.279160    4544 scope.go:122] "RemoveContainer" containerID="a59670c48701cef89ae80e744176158fafe263bb19e716e4b1a62e151911e04e"
Oct 30 19:06:18.291313 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.291289    4544 scope.go:122] "RemoveContainer" containerID="491b582a839a2c50de4f7231ed0bb51e306dc3094feb01f0a8d202c019d67d21"
Oct 30 19:06:18.291643 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:06:18.291596    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"491b582a839a2c50de4f7231ed0bb51e306dc3094feb01f0a8d202c019d67d21\": container with ID starting with 491b582a839a2c50de4f7231ed0bb51e306dc3094feb01f0a8d202c019d67d21 not found: ID does not exist" containerID="491b582a839a2c50de4f7231ed0bb51e306dc3094feb01f0a8d202c019d67d21"
Oct 30 19:06:18.291742 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.291653    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"491b582a839a2c50de4f7231ed0bb51e306dc3094feb01f0a8d202c019d67d21"} err="failed to get container status \"491b582a839a2c50de4f7231ed0bb51e306dc3094feb01f0a8d202c019d67d21\": rpc error: code = NotFound desc = could not find container \"491b582a839a2c50de4f7231ed0bb51e306dc3094feb01f0a8d202c019d67d21\": container with ID starting with 491b582a839a2c50de4f7231ed0bb51e306dc3094feb01f0a8d202c019d67d21 not found: ID does not exist"
Oct 30 19:06:18.291742 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.291673    4544 scope.go:122] "RemoveContainer" containerID="a59670c48701cef89ae80e744176158fafe263bb19e716e4b1a62e151911e04e"
Oct 30 19:06:18.291963 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:06:18.291944    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a59670c48701cef89ae80e744176158fafe263bb19e716e4b1a62e151911e04e\": container with ID starting with a59670c48701cef89ae80e744176158fafe263bb19e716e4b1a62e151911e04e not found: ID does not exist" containerID="a59670c48701cef89ae80e744176158fafe263bb19e716e4b1a62e151911e04e"
Oct 30 19:06:18.292025 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.291969    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a59670c48701cef89ae80e744176158fafe263bb19e716e4b1a62e151911e04e"} err="failed to get container status \"a59670c48701cef89ae80e744176158fafe263bb19e716e4b1a62e151911e04e\": rpc error: code = NotFound desc = could not find container \"a59670c48701cef89ae80e744176158fafe263bb19e716e4b1a62e151911e04e\": container with ID starting with a59670c48701cef89ae80e744176158fafe263bb19e716e4b1a62e151911e04e not found: ID does not exist"
Oct 30 19:06:18.292025 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.291984    4544 scope.go:122] "RemoveContainer" containerID="edbf6c2e5bb0b8e6ea666133fc8cc68d9852bdb2ae1aacb41f93293cdca3c13b"
Oct 30 19:06:18.630125 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.630092    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"]
Oct 30 19:06:18.633729 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.633702    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-jddlt"]
Oct 30 19:06:18.737301 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.737246    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"]
Oct 30 19:06:18.740960 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.740923    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-zhbks"]
Oct 30 19:06:18.852537 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.852503    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"]
Oct 30 19:06:18.856549 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.856526    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lbfsk"]
Oct 30 19:06:18.967421 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.967346    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"]
Oct 30 19:06:18.973352 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:18.973332    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-gm6wv"]
Oct 30 19:06:19.083986 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:19.083933    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"]
Oct 30 19:06:19.087566 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:19.087543    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-a1617e-openshift-default-75c6f6977c-n9lr2"]
Oct 30 19:06:19.198397 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:19.198363    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"]
Oct 30 19:06:19.203327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:19.203301    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-mw9sj"]
Oct 30 19:06:19.314720 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:19.314691    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"]
Oct 30 19:06:19.317621 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:19.317578    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-bvhlk"]
Oct 30 19:06:19.428471 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:19.428434    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"]
Oct 30 19:06:19.432275 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:19.432250    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ec8dab-openshift-default-6948fb4f79-lhsn7"]
Oct 30 19:06:20.604899 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:20.604866    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" path="/var/lib/kubelet/pods/0da27f8b-2d0d-4737-a02f-ab1e1d7d150b/volumes"
Oct 30 19:06:20.605494 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:20.605357    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="13a222a4-d62a-4078-bd12-c03a8996dc87" path="/var/lib/kubelet/pods/13a222a4-d62a-4078-bd12-c03a8996dc87/volumes"
Oct 30 19:06:20.605787 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:20.605771    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48229a00-0acf-4c2b-b14e-d0151692b2d1" path="/var/lib/kubelet/pods/48229a00-0acf-4c2b-b14e-d0151692b2d1/volumes"
Oct 30 19:06:20.606494 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:20.606477    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="64da4d53-629c-4015-aaf9-2e3765f9dc5c" path="/var/lib/kubelet/pods/64da4d53-629c-4015-aaf9-2e3765f9dc5c/volumes"
Oct 30 19:06:20.606917 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:20.606904    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71bee142-6883-471f-933d-18f2424a1bac" path="/var/lib/kubelet/pods/71bee142-6883-471f-933d-18f2424a1bac/volumes"
Oct 30 19:06:20.607312 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:20.607300    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" path="/var/lib/kubelet/pods/8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d/volumes"
Oct 30 19:06:20.608027 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:20.608011    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="94fb0221-0a90-421d-9504-80067ac621ca" path="/var/lib/kubelet/pods/94fb0221-0a90-421d-9504-80067ac621ca/volumes"
Oct 30 19:06:20.608430 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:20.608418    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c22154c8-6939-496a-b422-29a775dcbf57" path="/var/lib/kubelet/pods/c22154c8-6939-496a-b422-29a775dcbf57/volumes"
Oct 30 19:06:21.408290 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.408260    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"]
Oct 30 19:06:21.408949 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.408931    4544 cpu_manager.go:414] "containerMap: removing container" podUID="71bee142-6883-471f-933d-18f2424a1bac" containerName="istio-proxy"
Oct 30 19:06:21.409022 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.408951    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71bee142-6883-471f-933d-18f2424a1bac" containerName="istio-proxy"
Oct 30 19:06:21.409022 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.408964    4544 cpu_manager.go:414] "containerMap: removing container" podUID="48229a00-0acf-4c2b-b14e-d0151692b2d1" containerName="istio-proxy"
Oct 30 19:06:21.409022 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.408969    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48229a00-0acf-4c2b-b14e-d0151692b2d1" containerName="istio-proxy"
Oct 30 19:06:21.409022 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.408978    4544 cpu_manager.go:414] "containerMap: removing container" podUID="94fb0221-0a90-421d-9504-80067ac621ca" containerName="istio-proxy"
Oct 30 19:06:21.409022 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.408983    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94fb0221-0a90-421d-9504-80067ac621ca" containerName="istio-proxy"
Oct 30 19:06:21.409022 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409004    4544 cpu_manager.go:414] "containerMap: removing container" podUID="64da4d53-629c-4015-aaf9-2e3765f9dc5c" containerName="istio-proxy"
Oct 30 19:06:21.409022 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409008    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="64da4d53-629c-4015-aaf9-2e3765f9dc5c" containerName="istio-proxy"
Oct 30 19:06:21.409022 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409018    4544 cpu_manager.go:414] "containerMap: removing container" podUID="13a222a4-d62a-4078-bd12-c03a8996dc87" containerName="istio-proxy"
Oct 30 19:06:21.409022 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409023    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="13a222a4-d62a-4078-bd12-c03a8996dc87" containerName="istio-proxy"
Oct 30 19:06:21.409307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409036    4544 cpu_manager.go:414] "containerMap: removing container" podUID="c22154c8-6939-496a-b422-29a775dcbf57" containerName="istio-proxy"
Oct 30 19:06:21.409307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409041    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c22154c8-6939-496a-b422-29a775dcbf57" containerName="istio-proxy"
Oct 30 19:06:21.409307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409048    4544 cpu_manager.go:414] "containerMap: removing container" podUID="8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" containerName="istio-proxy"
Oct 30 19:06:21.409307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409051    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" containerName="istio-proxy"
Oct 30 19:06:21.409307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409058    4544 cpu_manager.go:414] "containerMap: removing container" podUID="0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" containerName="istio-proxy"
Oct 30 19:06:21.409307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409062    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" containerName="istio-proxy"
Oct 30 19:06:21.409307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409069    4544 cpu_manager.go:414] "containerMap: removing container" podUID="0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" containerName="istio-proxy"
Oct 30 19:06:21.409307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409072    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" containerName="istio-proxy"
Oct 30 19:06:21.409307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409166    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="13a222a4-d62a-4078-bd12-c03a8996dc87" containerName="istio-proxy"
Oct 30 19:06:21.409307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409174    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="c22154c8-6939-496a-b422-29a775dcbf57" containerName="istio-proxy"
Oct 30 19:06:21.409307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409181    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="8626c2ef-59e9-4fce-90a9-5eb28e6dfd4d" containerName="istio-proxy"
Oct 30 19:06:21.409307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409188    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" containerName="istio-proxy"
Oct 30 19:06:21.409307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409194    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="0da27f8b-2d0d-4737-a02f-ab1e1d7d150b" containerName="istio-proxy"
Oct 30 19:06:21.409307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409201    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="71bee142-6883-471f-933d-18f2424a1bac" containerName="istio-proxy"
Oct 30 19:06:21.409307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409208    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="48229a00-0acf-4c2b-b14e-d0151692b2d1" containerName="istio-proxy"
Oct 30 19:06:21.409307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409215    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="94fb0221-0a90-421d-9504-80067ac621ca" containerName="istio-proxy"
Oct 30 19:06:21.409307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.409223    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="64da4d53-629c-4015-aaf9-2e3765f9dc5c" containerName="istio-proxy"
Oct 30 19:06:21.414164 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.414148    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.481964 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.481933    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.481964 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.481964    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e14048da-861d-4530-851e-8417b6cbf8bb-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.482199 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.482016    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.482199 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.482109    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e14048da-861d-4530-851e-8417b6cbf8bb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.482199 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.482147    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjr4q\" (UniqueName: \"kubernetes.io/projected/e14048da-861d-4530-851e-8417b6cbf8bb-kube-api-access-xjr4q\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.482199 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.482188    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e14048da-861d-4530-851e-8417b6cbf8bb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.482362 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.482291    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.482413 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.482400    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.482448 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.482422    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.508448 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.508417    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"]
Oct 30 19:06:21.583699 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.583665    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.583699 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.583703    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.583905 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.583852    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.583905 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.583881    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e14048da-861d-4530-851e-8417b6cbf8bb-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.583979 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.583947    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.584031 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.583984    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e14048da-861d-4530-851e-8417b6cbf8bb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.584031 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.584014    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjr4q\" (UniqueName: \"kubernetes.io/projected/e14048da-861d-4530-851e-8417b6cbf8bb-kube-api-access-xjr4q\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.584131 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.584111    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.584185 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.584153    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e14048da-861d-4530-851e-8417b6cbf8bb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.584235 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.584188    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.584293 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.584244    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.584483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.584397    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.584622 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.584508    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.585053 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.585029    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e14048da-861d-4530-851e-8417b6cbf8bb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.586280 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.586255    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.586712 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.586694    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e14048da-861d-4530-851e-8417b6cbf8bb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.641378 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.641345    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e14048da-861d-4530-851e-8417b6cbf8bb-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.641725 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.641675    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjr4q\" (UniqueName: \"kubernetes.io/projected/e14048da-861d-4530-851e-8417b6cbf8bb-kube-api-access-xjr4q\") pod \"maas-default-gateway-openshift-default-75d7c785f-f22z9\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:21.728140 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:21.728060    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:22.156677 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:06:22.156597    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode14048da_861d_4530_851e_8417b6cbf8bb.slice/crio-47e964ac8aa0d40fd3f1e27ea5d5e680f9814d7e71a6055995242fa374b60f83 WatchSource:0}: Error finding container 47e964ac8aa0d40fd3f1e27ea5d5e680f9814d7e71a6055995242fa374b60f83: Status 404 returned error can't find the container with id 47e964ac8aa0d40fd3f1e27ea5d5e680f9814d7e71a6055995242fa374b60f83
Oct 30 19:06:22.158102 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:22.158074    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"]
Oct 30 19:06:22.159933 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:22.159896    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:06:22.160016 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:22.159985    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:06:22.160055 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:22.160026    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:06:22.229520 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:22.229487    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9" event={"ID":"e14048da-861d-4530-851e-8417b6cbf8bb","Type":"ContainerStarted","Data":"47e964ac8aa0d40fd3f1e27ea5d5e680f9814d7e71a6055995242fa374b60f83"}
Oct 30 19:06:23.237597 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:23.237557    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9" event={"ID":"e14048da-861d-4530-851e-8417b6cbf8bb","Type":"ContainerStarted","Data":"5a3f9d66bea7946fa68eb98db78807ce9666c883ad9fc152bd37e0e121245dfc"}
Oct 30 19:06:23.414683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:23.414590    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9" podStartSLOduration=2.414576838 podStartE2EDuration="2.414576838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:23.413231233 +0000 UTC m=+1780.939106329" watchObservedRunningTime="2026-10-30 19:06:23.414576838 +0000 UTC m=+1780.940451933"
Oct 30 19:06:23.728785 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:23.728753    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:23.730404 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:23.730373    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-f22z9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:06:23.730536 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:23.730427    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9" podUID="e14048da-861d-4530-851e-8417b6cbf8bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 19:06:24.728466 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:24.728433    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-f22z9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:06:24.728934 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:24.728497    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9" podUID="e14048da-861d-4530-851e-8417b6cbf8bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 19:06:25.728468 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:25.728432    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-f22z9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:06:25.728919 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:25.728499    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9" podUID="e14048da-861d-4530-851e-8417b6cbf8bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 19:06:26.728547 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:26.728507    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-f22z9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:06:26.728940 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:26.728565    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9" podUID="e14048da-861d-4530-851e-8417b6cbf8bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 19:06:27.729298 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:27.729262    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-f22z9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:06:27.729720 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:27.729315    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9" podUID="e14048da-861d-4530-851e-8417b6cbf8bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 19:06:28.728663 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:28.728592    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-f22z9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:06:28.728855 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:28.728706    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9" podUID="e14048da-861d-4530-851e-8417b6cbf8bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 19:06:29.728562 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:29.728525    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-f22z9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 19:06:29.729122 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:29.729089    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9" podUID="e14048da-861d-4530-851e-8417b6cbf8bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 19:06:31.786626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:31.785715    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-f22z9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:06:31.786626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:31.785800    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9" podUID="e14048da-861d-4530-851e-8417b6cbf8bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:06:31.872043 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:31.871994    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 19:06:31.915912 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:31.915712    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-zp4pg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:06:31.915912 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:31.915822    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg" podUID="5ccae868-24e4-456f-ad9a-58c17d714b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:06:31.916222 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:31.916204    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-j76r7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:06:31.916386 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:31.916363    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7" podUID="edd86ca1-466c-4e45-8998-61d7e94fae7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:06:31.964906 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:31.964873    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:31.982627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:31.967476    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:31.990145 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:31.990119    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:06:32.075087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:32.074988    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:06:32.075728 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:32.075696    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:06:32.129775 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:32.129727    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:06:33.548740 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:33.548706    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"]
Oct 30 19:06:33.552806 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:33.552786    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:06:33.610234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:33.610206    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"]
Oct 30 19:06:33.653683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:33.653650    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c\" (UID: \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:06:33.653846 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:33.653708    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c\" (UID: \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:06:33.653846 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:33.653798    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c\" (UID: \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:06:33.653968 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:33.653947    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dpwq\" (UniqueName: \"kubernetes.io/projected/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-kube-api-access-8dpwq\") pod \"maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c\" (UID: \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:06:33.654819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:33.654797    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-091e8f-serving-cert\""
Oct 30 19:06:33.755803 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:33.755772    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c\" (UID: \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:06:33.755977 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:33.755821    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c\" (UID: \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:06:33.755977 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:33.755846    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c\" (UID: \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:06:33.755977 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:06:33.755944    4544 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-serving-cert: secret "maas-api-e2e-shared-b-w1-091e8f-serving-cert" not found
Oct 30 19:06:33.755977 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:33.755950    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dpwq\" (UniqueName: \"kubernetes.io/projected/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-kube-api-access-8dpwq\") pod \"maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c\" (UID: \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:06:33.756226 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:06:33.756012    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-maas-api-tls podName:75bf6071-0e48-4e7e-b1d3-284f2c14e52e nodeName:}" failed. No retries permitted until 2026-10-30 19:06:34.255988706 +0000 UTC m=+1791.781863797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-maas-api-tls") pod "maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c" (UID: "75bf6071-0e48-4e7e-b1d3-284f2c14e52e") : secret "maas-api-e2e-shared-b-w1-091e8f-serving-cert" not found
Oct 30 19:06:33.756582 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:33.756555    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c\" (UID: \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:06:33.802671 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:33.801842    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c\" (UID: \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:06:33.831625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:33.825982    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dpwq\" (UniqueName: \"kubernetes.io/projected/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-kube-api-access-8dpwq\") pod \"maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c\" (UID: \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:06:34.280749 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:34.280709    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c\" (UID: \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:06:34.313816 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:34.313782    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c\" (UID: \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:06:34.502973 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:34.502927    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:06:35.019831 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:35.019798    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"]
Oct 30 19:06:35.468177 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:35.468136    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c" event={"ID":"75bf6071-0e48-4e7e-b1d3-284f2c14e52e","Type":"ContainerStarted","Data":"918b774b86ad27d2d711a0ab276edeb5c2e83a34a7f97d141e0ad43589a76449"}
Oct 30 19:06:35.600424 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:06:35.600392    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:06:36.478723 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:36.478671    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c" event={"ID":"75bf6071-0e48-4e7e-b1d3-284f2c14e52e","Type":"ContainerStarted","Data":"13c3de8698598028fb42a3eff8508dd7e2b301e4a3377cb05a85b9c80908e3d9"}
Oct 30 19:06:36.479327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:36.479299    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:06:36.596447 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:36.596368    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c" podStartSLOduration=2.615150779 podStartE2EDuration="3.596347877s" totalImagesPullingTime="981.197098ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:36.594694305 +0000 UTC m=+1794.120569402" watchObservedRunningTime="2026-10-30 19:06:36.596347877 +0000 UTC m=+1794.122222976"
Oct 30 19:06:40.513552 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:40.513460    4544 generic.go:356] "Generic (PLEG): container finished" podID="14497a85-b284-431d-930f-a0ac9274ab6c" containerID="5450673dbd1bdb87f3cdc593241e287b3e8243744a944de9c5e594cb18f1e157" exitCode=0
Oct 30 19:06:40.513552 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:40.513530    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4" event={"ID":"14497a85-b284-431d-930f-a0ac9274ab6c","Type":"ContainerDied","Data":"5450673dbd1bdb87f3cdc593241e287b3e8243744a944de9c5e594cb18f1e157"}
Oct 30 19:06:40.514071 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:40.514055    4544 scope.go:122] "RemoveContainer" containerID="5450673dbd1bdb87f3cdc593241e287b3e8243744a944de9c5e594cb18f1e157"
Oct 30 19:06:40.515717 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:40.515685    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:06:40.515821 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:40.515752    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:06:40.515821 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:40.515783    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:06:41.521486 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:41.521453    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4" event={"ID":"14497a85-b284-431d-930f-a0ac9274ab6c","Type":"ContainerStarted","Data":"9cf1e5617bc9ad999c364179612169a3b9e2f5d93b72254a64486e30690a974f"}
Oct 30 19:06:42.287827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:42.287790    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:06:42.289169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:42.289138    4544 patch_prober.go:29] interesting pod/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:06:42.289310 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:42.289203    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4" podUID="14497a85-b284-431d-930f-a0ac9274ab6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 19:06:42.495429 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:42.495399    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:06:42.837851 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:42.837823    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 19:06:42.840892 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:42.840869    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 19:06:43.286627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:43.285985    4544 patch_prober.go:29] interesting pod/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:06:43.286627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:43.286042    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4" podUID="14497a85-b284-431d-930f-a0ac9274ab6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 19:06:44.294652 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.289974    4544 patch_prober.go:29] interesting pod/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:06:44.294652 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.290037    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4" podUID="14497a85-b284-431d-930f-a0ac9274ab6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 19:06:44.804724 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.804689    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"]
Oct 30 19:06:44.813106 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.811815    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.852209 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.852177    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-token\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.852390 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.852220    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-workload-certs\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.852390 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.852313    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-workload-socket\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.852390 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.852361    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-credential-socket\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.852548 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.852451    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-data\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.852629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.852544    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-podinfo\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.852629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.852579    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klts9\" (UniqueName: \"kubernetes.io/projected/fe43caaf-eb11-4b75-a185-66ab33940e27-kube-api-access-klts9\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.852744 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.852707    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe43caaf-eb11-4b75-a185-66ab33940e27-istiod-ca-cert\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.852844 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.852824    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-envoy\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.899844 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.899812    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-05565d-openshift-default-dockercfg-vch5d\""
Oct 30 19:06:44.953701 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.953666    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"]
Oct 30 19:06:44.954253 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.954218    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-envoy\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.954365 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.954346    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-token\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.954435 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.954389    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-workload-certs\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.954435 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.954422    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-workload-socket\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.954545 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.954449    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-credential-socket\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.954545 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.954500    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-data\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.954706 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.954625    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-podinfo\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.954706 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.954679    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klts9\" (UniqueName: \"kubernetes.io/projected/fe43caaf-eb11-4b75-a185-66ab33940e27-kube-api-access-klts9\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.954942 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.954772    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe43caaf-eb11-4b75-a185-66ab33940e27-istiod-ca-cert\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.955715 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.955688    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe43caaf-eb11-4b75-a185-66ab33940e27-istiod-ca-cert\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.956006 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.955983    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-workload-socket\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.956050 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.956008    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-credential-socket\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.956269 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.956249    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-data\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:44.960625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:44.956659    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-workload-certs\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:45.012427 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:45.002972    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-podinfo\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:45.024766 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:45.023024    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klts9\" (UniqueName: \"kubernetes.io/projected/fe43caaf-eb11-4b75-a185-66ab33940e27-kube-api-access-klts9\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:45.041475 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:45.041443    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-envoy\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:45.041728 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:45.041555    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-token\") pod \"e2e-api-a-05565d-openshift-default-9c499798c-dmvl2\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") " pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:45.129674 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:45.129547    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:45.304125 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:45.303159    4544 patch_prober.go:29] interesting pod/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:06:45.304125 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:45.303217    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4" podUID="14497a85-b284-431d-930f-a0ac9274ab6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 19:06:45.598235 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:06:45.598201    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe43caaf_eb11_4b75_a185_66ab33940e27.slice/crio-bb28c8b8c5fe6961b306a2666fdf58242508a30fef9baab3716e3879c00c68c5 WatchSource:0}: Error finding container bb28c8b8c5fe6961b306a2666fdf58242508a30fef9baab3716e3879c00c68c5: Status 404 returned error can't find the container with id bb28c8b8c5fe6961b306a2666fdf58242508a30fef9baab3716e3879c00c68c5
Oct 30 19:06:45.600062 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:45.600040    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"]
Oct 30 19:06:45.601186 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:45.601151    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:06:45.601265 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:45.601221    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:06:45.601265 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:45.601249    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:06:46.287968 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:46.287927    4544 patch_prober.go:29] interesting pod/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:06:46.288148 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:46.287978    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4" podUID="14497a85-b284-431d-930f-a0ac9274ab6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 19:06:46.319634 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:46.319574    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:06:46.555444 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:46.555359    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2" event={"ID":"fe43caaf-eb11-4b75-a185-66ab33940e27","Type":"ContainerStarted","Data":"46891d3a4bed07102f87a20dde1eab6e80588d4a99be77e94bcd8b1fdc6cf09e"}
Oct 30 19:06:46.555444 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:46.555410    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2" event={"ID":"fe43caaf-eb11-4b75-a185-66ab33940e27","Type":"ContainerStarted","Data":"bb28c8b8c5fe6961b306a2666fdf58242508a30fef9baab3716e3879c00c68c5"}
Oct 30 19:06:46.684629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:46.684010    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2" podStartSLOduration=2.6839887559999998 podStartE2EDuration="2.683988756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:46.676062402 +0000 UTC m=+1804.201937536" watchObservedRunningTime="2026-10-30 19:06:46.683988756 +0000 UTC m=+1804.209863853"
Oct 30 19:06:47.130433 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:47.130396    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:47.137987 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:47.137960    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:47.290486 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:47.290456    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:06:47.291662 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:47.291640    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:06:47.577638 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:47.576483    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:47.577638 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:47.577267    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:06:51.653722 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.653688    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"]
Oct 30 19:06:51.658190 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.658168    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.797363 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.797314    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.797551 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.797449    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0686c324-b270-432b-9dbb-8d6ad419a957-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.797551 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.797546    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v6h6\" (UniqueName: \"kubernetes.io/projected/0686c324-b270-432b-9dbb-8d6ad419a957-kube-api-access-2v6h6\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.797702 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.797578    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.797764 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.797701    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0686c324-b270-432b-9dbb-8d6ad419a957-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.797764 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.797743    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0686c324-b270-432b-9dbb-8d6ad419a957-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.797868 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.797848    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.797924 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.797878    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.797924 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.797911    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.856782 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.856742    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"]
Oct 30 19:06:51.899423 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.899372    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.899677 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.899428    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.899677 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.899474    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.899677 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.899575    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.899677 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.899668    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0686c324-b270-432b-9dbb-8d6ad419a957-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.901415 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.899743    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v6h6\" (UniqueName: \"kubernetes.io/projected/0686c324-b270-432b-9dbb-8d6ad419a957-kube-api-access-2v6h6\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.901415 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.899780    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.901415 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.899839    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0686c324-b270-432b-9dbb-8d6ad419a957-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.901415 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.899896    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0686c324-b270-432b-9dbb-8d6ad419a957-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.901415 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.901226    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.901705 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.901511    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.904224 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.904131    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.904336 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.904229    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.904820 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.904795    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0686c324-b270-432b-9dbb-8d6ad419a957-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.905947 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.905923    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0686c324-b270-432b-9dbb-8d6ad419a957-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.907025 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.907005    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.962168 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.962130    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0686c324-b270-432b-9dbb-8d6ad419a957-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.962304 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.962253    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v6h6\" (UniqueName: \"kubernetes.io/projected/0686c324-b270-432b-9dbb-8d6ad419a957-kube-api-access-2v6h6\") pod \"maas-default-gateway-openshift-default-75d7c785f-2jstd\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:51.973411 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:51.973390    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:52.400359 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:06:52.400327    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0686c324_b270_432b_9dbb_8d6ad419a957.slice/crio-05a2873b0c04aee33f3d618a972aff59d32b8ca113cbb4761eb33b288c1ef0e4 WatchSource:0}: Error finding container 05a2873b0c04aee33f3d618a972aff59d32b8ca113cbb4761eb33b288c1ef0e4: Status 404 returned error can't find the container with id 05a2873b0c04aee33f3d618a972aff59d32b8ca113cbb4761eb33b288c1ef0e4
Oct 30 19:06:52.403255 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:52.403216    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:06:52.403372 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:52.403290    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:06:52.403372 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:52.403321    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:06:52.404938 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:52.404914    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"]
Oct 30 19:06:52.619978 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:52.619938    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd" event={"ID":"0686c324-b270-432b-9dbb-8d6ad419a957","Type":"ContainerStarted","Data":"26fb1c7b93a9e8105277b79fce53caf721944fe6e5fb87f9b8f59aab1a3aa7f5"}
Oct 30 19:06:52.620143 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:52.619988    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd" event={"ID":"0686c324-b270-432b-9dbb-8d6ad419a957","Type":"ContainerStarted","Data":"05a2873b0c04aee33f3d618a972aff59d32b8ca113cbb4761eb33b288c1ef0e4"}
Oct 30 19:06:52.754025 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:52.753886    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd" podStartSLOduration=1.753865928 podStartE2EDuration="1.753865928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:52.75189652 +0000 UTC m=+1810.277771617" watchObservedRunningTime="2026-10-30 19:06:52.753865928 +0000 UTC m=+1810.279741024"
Oct 30 19:06:52.974379 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:52.974341    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:06:53.002067 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.002026    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r"]
Oct 30 19:06:53.007885 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.007674    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r"
Oct 30 19:06:53.067573 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.067536    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-05565d\""
Oct 30 19:06:53.072961 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.072930    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-05565d-dockercfg-zhx77\""
Oct 30 19:06:53.098626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.091093    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r"]
Oct 30 19:06:53.206664 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.194968    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s54tx\" (UniqueName: \"kubernetes.io/projected/7482faa1-c2d0-4c2d-8998-b71343b0a141-kube-api-access-s54tx\") pod \"payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r\" (UID: \"7482faa1-c2d0-4c2d-8998-b71343b0a141\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r"
Oct 30 19:06:53.206664 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.195069    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7482faa1-c2d0-4c2d-8998-b71343b0a141-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r\" (UID: \"7482faa1-c2d0-4c2d-8998-b71343b0a141\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r"
Oct 30 19:06:53.242774 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.238666    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h"]
Oct 30 19:06:53.244270 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.244241    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h"
Oct 30 19:06:53.252238 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.252195    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h"]
Oct 30 19:06:53.301887 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.301739    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f117897d-7353-4281-909c-a475e208036d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h\" (UID: \"f117897d-7353-4281-909c-a475e208036d\") " pod="openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h"
Oct 30 19:06:53.306965 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.302260    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s54tx\" (UniqueName: \"kubernetes.io/projected/7482faa1-c2d0-4c2d-8998-b71343b0a141-kube-api-access-s54tx\") pod \"payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r\" (UID: \"7482faa1-c2d0-4c2d-8998-b71343b0a141\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r"
Oct 30 19:06:53.306965 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.302381    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7482faa1-c2d0-4c2d-8998-b71343b0a141-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r\" (UID: \"7482faa1-c2d0-4c2d-8998-b71343b0a141\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r"
Oct 30 19:06:53.306965 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.302458    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmsb5\" (UniqueName: \"kubernetes.io/projected/f117897d-7353-4281-909c-a475e208036d-kube-api-access-fmsb5\") pod \"payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h\" (UID: \"f117897d-7353-4281-909c-a475e208036d\") " pod="openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h"
Oct 30 19:06:53.306965 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.305649    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7482faa1-c2d0-4c2d-8998-b71343b0a141-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r\" (UID: \"7482faa1-c2d0-4c2d-8998-b71343b0a141\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r"
Oct 30 19:06:53.400628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.371094    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s54tx\" (UniqueName: \"kubernetes.io/projected/7482faa1-c2d0-4c2d-8998-b71343b0a141-kube-api-access-s54tx\") pod \"payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r\" (UID: \"7482faa1-c2d0-4c2d-8998-b71343b0a141\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r"
Oct 30 19:06:53.405352 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.404582    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f117897d-7353-4281-909c-a475e208036d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h\" (UID: \"f117897d-7353-4281-909c-a475e208036d\") " pod="openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h"
Oct 30 19:06:53.405478 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.405320    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f117897d-7353-4281-909c-a475e208036d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h\" (UID: \"f117897d-7353-4281-909c-a475e208036d\") " pod="openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h"
Oct 30 19:06:53.405551 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.405539    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmsb5\" (UniqueName: \"kubernetes.io/projected/f117897d-7353-4281-909c-a475e208036d-kube-api-access-fmsb5\") pod \"payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h\" (UID: \"f117897d-7353-4281-909c-a475e208036d\") " pod="openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h"
Oct 30 19:06:53.481626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.466451    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmsb5\" (UniqueName: \"kubernetes.io/projected/f117897d-7353-4281-909c-a475e208036d-kube-api-access-fmsb5\") pod \"payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h\" (UID: \"f117897d-7353-4281-909c-a475e208036d\") " pod="openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h"
Oct 30 19:06:53.578674 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.577913    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h"
Oct 30 19:06:53.649893 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.649661    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r"
Oct 30 19:06:53.983685 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.981738    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h"]
Oct 30 19:06:53.996552 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.996496    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-2jstd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:06:53.996706 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:53.996633    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd" podUID="0686c324-b270-432b-9dbb-8d6ad419a957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 19:06:54.116734 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:54.115525    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r"]
Oct 30 19:06:54.637395 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:54.637361    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r" event={"ID":"7482faa1-c2d0-4c2d-8998-b71343b0a141","Type":"ContainerStarted","Data":"b2dc66d236f58b649a8ed687354cffaabd406d80e8355d29be34b4df30cf6bc3"}
Oct 30 19:06:54.637587 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:54.637418    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r"
Oct 30 19:06:54.637587 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:54.637444    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r" event={"ID":"7482faa1-c2d0-4c2d-8998-b71343b0a141","Type":"ContainerStarted","Data":"8c4f3c519007d1a37cacbd41e64518a79335cc6fb5a31e89ca545591661e6837"}
Oct 30 19:06:54.639102 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:54.639066    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h" event={"ID":"f117897d-7353-4281-909c-a475e208036d","Type":"ContainerStarted","Data":"c0cad08639fbc527ba4edcc5f52b4d6e375199290ac9102c468d5a41d6fd75b0"}
Oct 30 19:06:54.639215 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:54.639119    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h" event={"ID":"f117897d-7353-4281-909c-a475e208036d","Type":"ContainerStarted","Data":"2c4db7bfd832fc34e1bb7fcee3a5efb688d8d6bc502c033e14ffe0a074077373"}
Oct 30 19:06:54.639273 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:54.639208    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h"
Oct 30 19:06:54.903196 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:54.903082    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r" podStartSLOduration=2.9030626010000002 podStartE2EDuration="2.903062601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:54.78097134 +0000 UTC m=+1812.306846438" watchObservedRunningTime="2026-10-30 19:06:54.903062601 +0000 UTC m=+1812.428937698"
Oct 30 19:06:54.981629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:54.975753    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-2jstd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:06:54.981629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:54.975815    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd" podUID="0686c324-b270-432b-9dbb-8d6ad419a957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 19:06:55.987672 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:55.980012    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-2jstd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:06:55.987672 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:55.980077    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd" podUID="0686c324-b270-432b-9dbb-8d6ad419a957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 19:06:56.981738 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:56.981696    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-2jstd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:06:56.982001 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:56.981749    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd" podUID="0686c324-b270-432b-9dbb-8d6ad419a957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 19:06:57.976001 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:57.975960    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-2jstd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:06:57.976589 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:57.976554    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd" podUID="0686c324-b270-432b-9dbb-8d6ad419a957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 19:06:58.974036 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:58.973995    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-2jstd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:06:58.974195 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:58.974058    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd" podUID="0686c324-b270-432b-9dbb-8d6ad419a957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 19:06:59.974639 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:59.974587    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-2jstd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:06:59.975053 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:06:59.974682    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd" podUID="0686c324-b270-432b-9dbb-8d6ad419a957" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.100:15021/healthz/ready\": dial tcp 10.133.0.100:15021: connect: connection refused"
Oct 30 19:07:01.361628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:01.357945    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:07:01.361628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:01.358514    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:07:01.361628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:01.359237    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:07:01.581965 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:01.581900    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h" podStartSLOduration=8.58188053 podStartE2EDuration="8.58188053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:54.90698978 +0000 UTC m=+1812.432864860" watchObservedRunningTime="2026-10-30 19:07:01.58188053 +0000 UTC m=+1819.107755627"
Oct 30 19:07:04.399581 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.399541    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"]
Oct 30 19:07:04.400007 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.399850    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g" podUID="a0a84f39-5760-4c04-af47-38ae3fa87a29" containerName="maas-api" containerID="cri-o://f4647b78b4c3cf9b4e0b25c05c56f1b48c03e9bcd979b21c71989d34c62c17a6" gracePeriod=30
Oct 30 19:07:04.733506 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.733473    4544 generic.go:356] "Generic (PLEG): container finished" podID="a0a84f39-5760-4c04-af47-38ae3fa87a29" containerID="f4647b78b4c3cf9b4e0b25c05c56f1b48c03e9bcd979b21c71989d34c62c17a6" exitCode=0
Oct 30 19:07:04.733669 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.733551    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g" event={"ID":"a0a84f39-5760-4c04-af47-38ae3fa87a29","Type":"ContainerDied","Data":"f4647b78b4c3cf9b4e0b25c05c56f1b48c03e9bcd979b21c71989d34c62c17a6"}
Oct 30 19:07:04.733669 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.733636    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g" event={"ID":"a0a84f39-5760-4c04-af47-38ae3fa87a29","Type":"ContainerDied","Data":"03d1062eb4b8a3ec47a712aea68ad39d298dc926ea6f7ed88ff2a8c022872fd4"}
Oct 30 19:07:04.733669 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.733664    4544 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03d1062eb4b8a3ec47a712aea68ad39d298dc926ea6f7ed88ff2a8c022872fd4"
Oct 30 19:07:04.745433 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.745410    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:07:04.821935 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.821902    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a0a84f39-5760-4c04-af47-38ae3fa87a29-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0a84f39-5760-4c04-af47-38ae3fa87a29-maas-api-tls\") pod \"a0a84f39-5760-4c04-af47-38ae3fa87a29\" (UID: \"a0a84f39-5760-4c04-af47-38ae3fa87a29\") "
Oct 30 19:07:04.822125 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.821954    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0a84f39-5760-4c04-af47-38ae3fa87a29-kube-api-access-z9snz\" (UniqueName: \"kubernetes.io/projected/a0a84f39-5760-4c04-af47-38ae3fa87a29-kube-api-access-z9snz\") pod \"a0a84f39-5760-4c04-af47-38ae3fa87a29\" (UID: \"a0a84f39-5760-4c04-af47-38ae3fa87a29\") "
Oct 30 19:07:04.822125 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.822054    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a0a84f39-5760-4c04-af47-38ae3fa87a29-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a84f39-5760-4c04-af47-38ae3fa87a29-openshift-service-ca\") pod \"a0a84f39-5760-4c04-af47-38ae3fa87a29\" (UID: \"a0a84f39-5760-4c04-af47-38ae3fa87a29\") "
Oct 30 19:07:04.822223 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.822195    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a0a84f39-5760-4c04-af47-38ae3fa87a29-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0a84f39-5760-4c04-af47-38ae3fa87a29-metrics-certs\") pod \"a0a84f39-5760-4c04-af47-38ae3fa87a29\" (UID: \"a0a84f39-5760-4c04-af47-38ae3fa87a29\") "
Oct 30 19:07:04.822551 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.822523    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0a84f39-5760-4c04-af47-38ae3fa87a29-openshift-service-ca" pod "a0a84f39-5760-4c04-af47-38ae3fa87a29" (UID: "a0a84f39-5760-4c04-af47-38ae3fa87a29"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:07:04.822918 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.822896    4544 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0a84f39-5760-4c04-af47-38ae3fa87a29-openshift-service-ca\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:04.828763 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.828735    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0a84f39-5760-4c04-af47-38ae3fa87a29-metrics-certs" pod "a0a84f39-5760-4c04-af47-38ae3fa87a29" (UID: "a0a84f39-5760-4c04-af47-38ae3fa87a29"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:07:04.828763 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.828750    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0a84f39-5760-4c04-af47-38ae3fa87a29-kube-api-access-z9snz" pod "a0a84f39-5760-4c04-af47-38ae3fa87a29" (UID: "a0a84f39-5760-4c04-af47-38ae3fa87a29"). InnerVolumeSpecName "kube-api-access-z9snz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:04.828905 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.828755    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0a84f39-5760-4c04-af47-38ae3fa87a29-maas-api-tls" pod "a0a84f39-5760-4c04-af47-38ae3fa87a29" (UID: "a0a84f39-5760-4c04-af47-38ae3fa87a29"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:07:04.924769 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.924731    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9snz\" (UniqueName: \"kubernetes.io/projected/a0a84f39-5760-4c04-af47-38ae3fa87a29-kube-api-access-z9snz\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:04.924769 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.924763    4544 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0a84f39-5760-4c04-af47-38ae3fa87a29-metrics-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:04.924769 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:04.924775    4544 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a0a84f39-5760-4c04-af47-38ae3fa87a29-maas-api-tls\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:05.593884 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:07:05.593846    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:07:05.649823 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:05.649794    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h"
Oct 30 19:07:05.649979 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:05.649870    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r"
Oct 30 19:07:05.740795 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:05.740765    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"
Oct 30 19:07:05.999895 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:05.999807    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"]
Oct 30 19:07:06.003225 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:06.003201    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4cef683f-5c7cbfdbbb-v9s4g"]
Oct 30 19:07:06.623586 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:06.623551    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0a84f39-5760-4c04-af47-38ae3fa87a29" path="/var/lib/kubelet/pods/a0a84f39-5760-4c04-af47-38ae3fa87a29/volumes"
Oct 30 19:07:08.963560 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:08.963525    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"]
Oct 30 19:07:08.964222 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:08.964207    4544 cpu_manager.go:414] "containerMap: removing container" podUID="a0a84f39-5760-4c04-af47-38ae3fa87a29" containerName="maas-api"
Oct 30 19:07:08.964279 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:08.964224    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0a84f39-5760-4c04-af47-38ae3fa87a29" containerName="maas-api"
Oct 30 19:07:08.964361 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:08.964347    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0a84f39-5760-4c04-af47-38ae3fa87a29" containerName="maas-api"
Oct 30 19:07:08.970218 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:08.970195    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:08.991689 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:08.991196    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7xxp\" (UniqueName: \"kubernetes.io/projected/729b4f97-7d22-49ae-ac1b-4528af3b5242-kube-api-access-p7xxp\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:08.991689 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:08.991256    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-data\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:08.991689 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:08.991299    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-envoy\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:08.991689 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:08.991385    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-credential-socket\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:08.991689 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:08.991415    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-workload-socket\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:08.991689 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:08.991449    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-workload-certs\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:08.991689 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:08.991638    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/729b4f97-7d22-49ae-ac1b-4528af3b5242-istiod-ca-cert\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:08.991689 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:08.991682    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-podinfo\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:08.992130 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:08.991721    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-token\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.035700 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.035663    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"]
Oct 30 19:07:09.065204 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.065168    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-2ef49a-openshift-default-dockercfg-bpc79\""
Oct 30 19:07:09.097491 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.097370    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-data\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.097491 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.097420    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-envoy\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.097491 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.097476    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-credential-socket\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.097491 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.097496    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-workload-socket\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.098023 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.097520    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-workload-certs\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.098023 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.097649    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/729b4f97-7d22-49ae-ac1b-4528af3b5242-istiod-ca-cert\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.098023 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.097744    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-podinfo\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.098023 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.097786    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-token\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.098023 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.097897    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7xxp\" (UniqueName: \"kubernetes.io/projected/729b4f97-7d22-49ae-ac1b-4528af3b5242-kube-api-access-p7xxp\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.105911 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.098823    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-data\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.105911 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.099681    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-workload-certs\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.105911 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.099957    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-credential-socket\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.105911 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.100205    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-workload-socket\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.105911 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.101698    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/729b4f97-7d22-49ae-ac1b-4528af3b5242-istiod-ca-cert\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.119030 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.118498    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-envoy\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.120737 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.120707    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-podinfo\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.171363 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.157876    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7xxp\" (UniqueName: \"kubernetes.io/projected/729b4f97-7d22-49ae-ac1b-4528af3b5242-kube-api-access-p7xxp\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.174152 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.174074    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-token\") pod \"e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") " pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.307096 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.307060    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:09.807565 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.807375    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"]
Oct 30 19:07:09.830168 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:07:09.830126    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod729b4f97_7d22_49ae_ac1b_4528af3b5242.slice/crio-3a1c72389702dd4813029c8712460795bd9bc9d819c7be15da8e474f86aca35b WatchSource:0}: Error finding container 3a1c72389702dd4813029c8712460795bd9bc9d819c7be15da8e474f86aca35b: Status 404 returned error can't find the container with id 3a1c72389702dd4813029c8712460795bd9bc9d819c7be15da8e474f86aca35b
Oct 30 19:07:09.832959 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.832925    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:07:09.833048 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.833001    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:07:09.833048 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:09.833029    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:07:10.788551 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:10.788513    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb" event={"ID":"729b4f97-7d22-49ae-ac1b-4528af3b5242","Type":"ContainerStarted","Data":"571a3de3132365c3dc0b58d04e53ce7ad88e4c525ed948d1fabac6985a99a850"}
Oct 30 19:07:10.789058 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:10.788577    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb" event={"ID":"729b4f97-7d22-49ae-ac1b-4528af3b5242","Type":"ContainerStarted","Data":"3a1c72389702dd4813029c8712460795bd9bc9d819c7be15da8e474f86aca35b"}
Oct 30 19:07:11.307833 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:11.307800    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:11.314096 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:11.314071    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:11.434923 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:11.434870    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb" podStartSLOduration=3.434856016 podStartE2EDuration="3.434856016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:11.004578143 +0000 UTC m=+1828.530453238" watchObservedRunningTime="2026-10-30 19:07:11.434856016 +0000 UTC m=+1828.960731111"
Oct 30 19:07:11.795672 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:11.795641    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:11.796817 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:11.796796    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:07:17.433040 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.425194    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl"]
Oct 30 19:07:17.433040 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.432062    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl"
Oct 30 19:07:17.487849 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.487795    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl"]
Oct 30 19:07:17.527982 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.526705    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18c8ec61-e79e-4370-a890-477e9ba68f10-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl\" (UID: \"18c8ec61-e79e-4370-a890-477e9ba68f10\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl"
Oct 30 19:07:17.527982 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.527045    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shp78\" (UniqueName: \"kubernetes.io/projected/18c8ec61-e79e-4370-a890-477e9ba68f10-kube-api-access-shp78\") pod \"payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl\" (UID: \"18c8ec61-e79e-4370-a890-477e9ba68f10\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl"
Oct 30 19:07:17.553657 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.551925    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-2ef49a\""
Oct 30 19:07:17.624475 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.624434    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz"]
Oct 30 19:07:17.648634 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.635308    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shp78\" (UniqueName: \"kubernetes.io/projected/18c8ec61-e79e-4370-a890-477e9ba68f10-kube-api-access-shp78\") pod \"payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl\" (UID: \"18c8ec61-e79e-4370-a890-477e9ba68f10\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl"
Oct 30 19:07:17.648634 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.635415    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18c8ec61-e79e-4370-a890-477e9ba68f10-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl\" (UID: \"18c8ec61-e79e-4370-a890-477e9ba68f10\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl"
Oct 30 19:07:17.648634 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.636784    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18c8ec61-e79e-4370-a890-477e9ba68f10-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl\" (UID: \"18c8ec61-e79e-4370-a890-477e9ba68f10\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl"
Oct 30 19:07:17.648634 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.638542    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz"
Oct 30 19:07:17.681936 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.681892    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz"]
Oct 30 19:07:17.714041 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.713957    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-2ef49a-dockercfg-7csjc\""
Oct 30 19:07:17.728270 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.728237    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shp78\" (UniqueName: \"kubernetes.io/projected/18c8ec61-e79e-4370-a890-477e9ba68f10-kube-api-access-shp78\") pod \"payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl\" (UID: \"18c8ec61-e79e-4370-a890-477e9ba68f10\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl"
Oct 30 19:07:17.768565 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.768519    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl"
Oct 30 19:07:17.771490 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.771451    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vzdl\" (UniqueName: \"kubernetes.io/projected/a5dea2af-18e6-458e-9e34-561bb06ba3f6-kube-api-access-2vzdl\") pod \"payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz\" (UID: \"a5dea2af-18e6-458e-9e34-561bb06ba3f6\") " pod="openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz"
Oct 30 19:07:17.774015 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.773980    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5dea2af-18e6-458e-9e34-561bb06ba3f6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz\" (UID: \"a5dea2af-18e6-458e-9e34-561bb06ba3f6\") " pod="openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz"
Oct 30 19:07:17.877120 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.877081    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5dea2af-18e6-458e-9e34-561bb06ba3f6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz\" (UID: \"a5dea2af-18e6-458e-9e34-561bb06ba3f6\") " pod="openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz"
Oct 30 19:07:17.877849 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.877805    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vzdl\" (UniqueName: \"kubernetes.io/projected/a5dea2af-18e6-458e-9e34-561bb06ba3f6-kube-api-access-2vzdl\") pod \"payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz\" (UID: \"a5dea2af-18e6-458e-9e34-561bb06ba3f6\") " pod="openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz"
Oct 30 19:07:17.878248 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.878225    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5dea2af-18e6-458e-9e34-561bb06ba3f6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz\" (UID: \"a5dea2af-18e6-458e-9e34-561bb06ba3f6\") " pod="openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz"
Oct 30 19:07:17.941642 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.939552    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vzdl\" (UniqueName: \"kubernetes.io/projected/a5dea2af-18e6-458e-9e34-561bb06ba3f6-kube-api-access-2vzdl\") pod \"payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz\" (UID: \"a5dea2af-18e6-458e-9e34-561bb06ba3f6\") " pod="openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz"
Oct 30 19:07:17.990835 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:17.990757    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz"
Oct 30 19:07:18.193310 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:07:18.193267    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18c8ec61_e79e_4370_a890_477e9ba68f10.slice/crio-51050f88a686c13f19fb4d5bab101d168cf809361424218dc51bf5e101e4822e WatchSource:0}: Error finding container 51050f88a686c13f19fb4d5bab101d168cf809361424218dc51bf5e101e4822e: Status 404 returned error can't find the container with id 51050f88a686c13f19fb4d5bab101d168cf809361424218dc51bf5e101e4822e
Oct 30 19:07:18.196383 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:18.196357    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl"]
Oct 30 19:07:18.373150 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:07:18.372269    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5dea2af_18e6_458e_9e34_561bb06ba3f6.slice/crio-4a5f13872f01c3f912f6642fe0673f61a8487156af3f07bc19d2485e708340b5 WatchSource:0}: Error finding container 4a5f13872f01c3f912f6642fe0673f61a8487156af3f07bc19d2485e708340b5: Status 404 returned error can't find the container with id 4a5f13872f01c3f912f6642fe0673f61a8487156af3f07bc19d2485e708340b5
Oct 30 19:07:18.375595 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:18.375569    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz"]
Oct 30 19:07:18.859559 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:18.859507    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl" event={"ID":"18c8ec61-e79e-4370-a890-477e9ba68f10","Type":"ContainerStarted","Data":"79e91c40363f48c56c65f16d0303b13bd64a1a186a639f5d18950d90093f99af"}
Oct 30 19:07:18.860057 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:18.859577    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl" event={"ID":"18c8ec61-e79e-4370-a890-477e9ba68f10","Type":"ContainerStarted","Data":"51050f88a686c13f19fb4d5bab101d168cf809361424218dc51bf5e101e4822e"}
Oct 30 19:07:18.860057 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:18.859777    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl"
Oct 30 19:07:18.861333 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:18.861307    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz" event={"ID":"a5dea2af-18e6-458e-9e34-561bb06ba3f6","Type":"ContainerStarted","Data":"ff702f8226aefdf3d16a5d68782b5b84a4cf1137056c41947379ea0054211e9a"}
Oct 30 19:07:18.861450 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:18.861354    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz" event={"ID":"a5dea2af-18e6-458e-9e34-561bb06ba3f6","Type":"ContainerStarted","Data":"4a5f13872f01c3f912f6642fe0673f61a8487156af3f07bc19d2485e708340b5"}
Oct 30 19:07:18.861450 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:18.861386    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz"
Oct 30 19:07:18.976093 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:18.976030    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl" podStartSLOduration=1.976009699 podStartE2EDuration="1.976009699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:18.974238053 +0000 UTC m=+1836.500113148" watchObservedRunningTime="2026-10-30 19:07:18.976009699 +0000 UTC m=+1836.501884797"
Oct 30 19:07:19.112313 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:19.093258    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz" podStartSLOduration=2.093237319 podStartE2EDuration="2.093237319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:19.088828168 +0000 UTC m=+1836.614703265" watchObservedRunningTime="2026-10-30 19:07:19.093237319 +0000 UTC m=+1836.619112416"
Oct 30 19:07:21.886586 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:21.886552    4544 generic.go:356] "Generic (PLEG): container finished" podID="b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" containerID="89c0a630c5e2f5178593997064c91bf04429f044c020b803cb6ef3bb4c3ae237" exitCode=0
Oct 30 19:07:21.887133 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:21.886640    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb" event={"ID":"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e","Type":"ContainerDied","Data":"89c0a630c5e2f5178593997064c91bf04429f044c020b803cb6ef3bb4c3ae237"}
Oct 30 19:07:21.887214 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:21.887176    4544 scope.go:122] "RemoveContainer" containerID="89c0a630c5e2f5178593997064c91bf04429f044c020b803cb6ef3bb4c3ae237"
Oct 30 19:07:21.889169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:21.889133    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:07:21.889305 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:21.889224    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:07:21.889305 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:21.889268    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:07:22.909074 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:22.908941    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb" event={"ID":"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e","Type":"ContainerStarted","Data":"c9a86d092f980a55d0751a71348126aa7dfc07fee891106feb3f137a7c14700e"}
Oct 30 19:07:23.744938 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:23.744903    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:07:23.746162 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:23.746130    4544 patch_prober.go:29] interesting pod/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:07:23.746307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:23.746188    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb" podUID="b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 19:07:24.745554 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:24.745515    4544 patch_prober.go:29] interesting pod/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:07:24.745976 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:24.745567    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb" podUID="b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 19:07:25.744765 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:25.744730    4544 patch_prober.go:29] interesting pod/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:07:25.744938 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:25.744783    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb" podUID="b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 19:07:26.630918 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:26.630886    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:07:26.745068 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:26.745040    4544 patch_prober.go:29] interesting pod/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:07:26.745233 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:26.745091    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb" podUID="b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 19:07:27.745043 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:27.745004    4544 patch_prober.go:29] interesting pod/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 19:07:27.745518 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:27.745061    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb" podUID="b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 19:07:27.998258 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:27.998168    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"]
Oct 30 19:07:27.998409 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:27.998357    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw" podUID="282d4bba-4530-4754-906f-f743b011ab66" containerName="istio-proxy" containerID="cri-o://bb4c3ad52dd097ccb283d270f5c51e9d338ccedee35a9f8a6bb5ab5997ca79f0" gracePeriod=30
Oct 30 19:07:28.750571 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:28.750529    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:07:28.751836 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:28.751811    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:07:29.872286 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:29.872253    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl"
Oct 30 19:07:29.872685 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:29.872342    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz"
Oct 30 19:07:33.288570 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.288543    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 19:07:33.308990 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.308960    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-workload-socket\") pod \"282d4bba-4530-4754-906f-f743b011ab66\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") "
Oct 30 19:07:33.309109 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.309059    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/282d4bba-4530-4754-906f-f743b011ab66-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/282d4bba-4530-4754-906f-f743b011ab66-istiod-ca-cert\") pod \"282d4bba-4530-4754-906f-f743b011ab66\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") "
Oct 30 19:07:33.309167 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.309120    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/282d4bba-4530-4754-906f-f743b011ab66-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/282d4bba-4530-4754-906f-f743b011ab66-istio-podinfo\") pod \"282d4bba-4530-4754-906f-f743b011ab66\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") "
Oct 30 19:07:33.309167 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.309141    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-credential-socket\") pod \"282d4bba-4530-4754-906f-f743b011ab66\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") "
Oct 30 19:07:33.309273 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.309197    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-workload-socket" pod "282d4bba-4530-4754-906f-f743b011ab66" (UID: "282d4bba-4530-4754-906f-f743b011ab66"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:33.309273 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.309251    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-istio-data\") pod \"282d4bba-4530-4754-906f-f743b011ab66\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") "
Oct 30 19:07:33.309380 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.309297    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-workload-certs\") pod \"282d4bba-4530-4754-906f-f743b011ab66\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") "
Oct 30 19:07:33.309436 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.309386    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-istio-envoy\") pod \"282d4bba-4530-4754-906f-f743b011ab66\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") "
Oct 30 19:07:33.309436 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.309387    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-credential-socket" pod "282d4bba-4530-4754-906f-f743b011ab66" (UID: "282d4bba-4530-4754-906f-f743b011ab66"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:33.309436 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.309419    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/282d4bba-4530-4754-906f-f743b011ab66-istio-token\" (UniqueName: \"kubernetes.io/projected/282d4bba-4530-4754-906f-f743b011ab66-istio-token\") pod \"282d4bba-4530-4754-906f-f743b011ab66\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") "
Oct 30 19:07:33.309592 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.309446    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/282d4bba-4530-4754-906f-f743b011ab66-kube-api-access-p9wbl\" (UniqueName: \"kubernetes.io/projected/282d4bba-4530-4754-906f-f743b011ab66-kube-api-access-p9wbl\") pod \"282d4bba-4530-4754-906f-f743b011ab66\" (UID: \"282d4bba-4530-4754-906f-f743b011ab66\") "
Oct 30 19:07:33.309592 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.309500    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-istio-data" pod "282d4bba-4530-4754-906f-f743b011ab66" (UID: "282d4bba-4530-4754-906f-f743b011ab66"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:33.309592 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.309536    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/282d4bba-4530-4754-906f-f743b011ab66-istiod-ca-cert" pod "282d4bba-4530-4754-906f-f743b011ab66" (UID: "282d4bba-4530-4754-906f-f743b011ab66"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:07:33.309778 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.309599    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-workload-certs" pod "282d4bba-4530-4754-906f-f743b011ab66" (UID: "282d4bba-4530-4754-906f-f743b011ab66"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:33.310178 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.310154    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:33.310287 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.310187    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:33.310287 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.310201    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:33.310287 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.310216    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:33.310287 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.310231    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/282d4bba-4530-4754-906f-f743b011ab66-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:33.311560 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.311522    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/282d4bba-4530-4754-906f-f743b011ab66-istio-podinfo" pod "282d4bba-4530-4754-906f-f743b011ab66" (UID: "282d4bba-4530-4754-906f-f743b011ab66"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:07:33.311700 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.311633    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/282d4bba-4530-4754-906f-f743b011ab66-istio-token" pod "282d4bba-4530-4754-906f-f743b011ab66" (UID: "282d4bba-4530-4754-906f-f743b011ab66"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:33.311700 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.311682    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/282d4bba-4530-4754-906f-f743b011ab66-kube-api-access-p9wbl" pod "282d4bba-4530-4754-906f-f743b011ab66" (UID: "282d4bba-4530-4754-906f-f743b011ab66"). InnerVolumeSpecName "kube-api-access-p9wbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:33.311904 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.311885    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-istio-envoy" pod "282d4bba-4530-4754-906f-f743b011ab66" (UID: "282d4bba-4530-4754-906f-f743b011ab66"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:33.412982 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.412878    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/282d4bba-4530-4754-906f-f743b011ab66-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:33.412982 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.412921    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/282d4bba-4530-4754-906f-f743b011ab66-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:33.412982 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.412937    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9wbl\" (UniqueName: \"kubernetes.io/projected/282d4bba-4530-4754-906f-f743b011ab66-kube-api-access-p9wbl\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:33.412982 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.412952    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/282d4bba-4530-4754-906f-f743b011ab66-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:33.985390 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.985354    4544 generic.go:356] "Generic (PLEG): container finished" podID="282d4bba-4530-4754-906f-f743b011ab66" containerID="bb4c3ad52dd097ccb283d270f5c51e9d338ccedee35a9f8a6bb5ab5997ca79f0" exitCode=0
Oct 30 19:07:33.985585 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.985402    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"
Oct 30 19:07:33.985585 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.985444    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw" event={"ID":"282d4bba-4530-4754-906f-f743b011ab66","Type":"ContainerDied","Data":"bb4c3ad52dd097ccb283d270f5c51e9d338ccedee35a9f8a6bb5ab5997ca79f0"}
Oct 30 19:07:33.985585 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.985499    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw" event={"ID":"282d4bba-4530-4754-906f-f743b011ab66","Type":"ContainerDied","Data":"fad63d922c71bb96634c38dc6dfef792c1f2ff421347309b1ba9cb87077ba42b"}
Oct 30 19:07:33.985585 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.985523    4544 scope.go:122] "RemoveContainer" containerID="bb4c3ad52dd097ccb283d270f5c51e9d338ccedee35a9f8a6bb5ab5997ca79f0"
Oct 30 19:07:33.996875 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.996852    4544 scope.go:122] "RemoveContainer" containerID="bb4c3ad52dd097ccb283d270f5c51e9d338ccedee35a9f8a6bb5ab5997ca79f0"
Oct 30 19:07:33.997171 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:07:33.997151    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb4c3ad52dd097ccb283d270f5c51e9d338ccedee35a9f8a6bb5ab5997ca79f0\": container with ID starting with bb4c3ad52dd097ccb283d270f5c51e9d338ccedee35a9f8a6bb5ab5997ca79f0 not found: ID does not exist" containerID="bb4c3ad52dd097ccb283d270f5c51e9d338ccedee35a9f8a6bb5ab5997ca79f0"
Oct 30 19:07:33.997220 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:33.997180    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb4c3ad52dd097ccb283d270f5c51e9d338ccedee35a9f8a6bb5ab5997ca79f0"} err="failed to get container status \"bb4c3ad52dd097ccb283d270f5c51e9d338ccedee35a9f8a6bb5ab5997ca79f0\": rpc error: code = NotFound desc = could not find container \"bb4c3ad52dd097ccb283d270f5c51e9d338ccedee35a9f8a6bb5ab5997ca79f0\": container with ID starting with bb4c3ad52dd097ccb283d270f5c51e9d338ccedee35a9f8a6bb5ab5997ca79f0 not found: ID does not exist"
Oct 30 19:07:34.169827 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:34.169788    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"]
Oct 30 19:07:34.210139 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:34.210110    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-r7llw"]
Oct 30 19:07:34.604172 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:34.604140    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="282d4bba-4530-4754-906f-f743b011ab66" path="/var/lib/kubelet/pods/282d4bba-4530-4754-906f-f743b011ab66/volumes"
Oct 30 19:07:35.531794 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:07:35.531760    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:07:45.082379 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:45.082343    4544 generic.go:356] "Generic (PLEG): container finished" podID="b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" containerID="0a71aabe4e79fb2504dd62e441b2e8649655cbade08bd02cea9011ad33f2c824" exitCode=0
Oct 30 19:07:45.082755 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:45.082416    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb" event={"ID":"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db","Type":"ContainerDied","Data":"0a71aabe4e79fb2504dd62e441b2e8649655cbade08bd02cea9011ad33f2c824"}
Oct 30 19:07:45.082929 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:45.082913    4544 scope.go:122] "RemoveContainer" containerID="0a71aabe4e79fb2504dd62e441b2e8649655cbade08bd02cea9011ad33f2c824"
Oct 30 19:07:45.084762 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:45.084738    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:07:45.084833 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:45.084810    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:07:45.084869 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:45.084847    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:07:46.098996 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:46.098958    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb" event={"ID":"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db","Type":"ContainerStarted","Data":"3c9e07561cde5edd626990a4690c7ce3593f36e0972f5d1fe09b2295c452c0db"}
Oct 30 19:07:46.928326 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:46.928286    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:07:46.929682 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:46.929653    4544 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 19:07:46.929816 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:46.929711    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb" podUID="b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 19:07:47.212890 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.212802    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"]
Oct 30 19:07:47.214473 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.214444    4544 cpu_manager.go:414] "containerMap: removing container" podUID="282d4bba-4530-4754-906f-f743b011ab66" containerName="istio-proxy"
Oct 30 19:07:47.214655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.214640    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="282d4bba-4530-4754-906f-f743b011ab66" containerName="istio-proxy"
Oct 30 19:07:47.214971 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.214957    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="282d4bba-4530-4754-906f-f743b011ab66" containerName="istio-proxy"
Oct 30 19:07:47.219477 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.219456    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.267430 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.267390    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"]
Oct 30 19:07:47.306852 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.278726    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-cfb433-openshift-default-dockercfg-wttm9\""
Oct 30 19:07:47.393659 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.331277    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-token\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.402732 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.402700    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-podinfo\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.402964 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.402942    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-credential-socket\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.403131 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.403116    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8ftp\" (UniqueName: \"kubernetes.io/projected/a70cfd95-bd84-4720-ae1a-630761e61fb1-kube-api-access-g8ftp\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.403255 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.403240    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a70cfd95-bd84-4720-ae1a-630761e61fb1-istiod-ca-cert\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.403450 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.403434    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-workload-certs\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.403627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.403592    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-workload-socket\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.403767 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.403751    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-data\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.404005 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.403990    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-envoy\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.507146 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.507045    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-token\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.507446 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.507421    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-podinfo\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.507596 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.507581    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-credential-socket\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.507772 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.507757    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8ftp\" (UniqueName: \"kubernetes.io/projected/a70cfd95-bd84-4720-ae1a-630761e61fb1-kube-api-access-g8ftp\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.507882 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.507868    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a70cfd95-bd84-4720-ae1a-630761e61fb1-istiod-ca-cert\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.508036 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.508016    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-workload-certs\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.508115 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.508071    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-workload-socket\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.508115 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.508102    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-data\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.508222 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.508199    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-envoy\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.511384 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.511085    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-workload-socket\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.511384 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.511133    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-workload-certs\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.511384 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.511308    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a70cfd95-bd84-4720-ae1a-630761e61fb1-istiod-ca-cert\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.511384 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.511347    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-data\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.511873 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.511839    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-credential-socket\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.513034 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.512938    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-envoy\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.514018 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.513998    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-podinfo\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.606625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.584355    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8ftp\" (UniqueName: \"kubernetes.io/projected/a70cfd95-bd84-4720-ae1a-630761e61fb1-kube-api-access-g8ftp\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.606625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.592185    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-token\") pod \"e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") " pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.834470 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.834434    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:07:47.875155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.873870    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:47.936662 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.929973    4544 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 19:07:47.936662 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:47.930046    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb" podUID="b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 19:07:48.430144 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:48.430108    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"]
Oct 30 19:07:48.449825 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:07:48.449275    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda70cfd95_bd84_4720_ae1a_630761e61fb1.slice/crio-7bfd7a3c8c5d0a688335638c8f9e0d9e1651c1a802849d845d1223b2939ae90b WatchSource:0}: Error finding container 7bfd7a3c8c5d0a688335638c8f9e0d9e1651c1a802849d845d1223b2939ae90b: Status 404 returned error can't find the container with id 7bfd7a3c8c5d0a688335638c8f9e0d9e1651c1a802849d845d1223b2939ae90b
Oct 30 19:07:48.452642 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:48.452561    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:07:48.452734 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:48.452666    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:07:48.452734 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:48.452700    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:07:48.928573 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:48.928534    4544 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 19:07:48.928784 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:48.928590    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb" podUID="b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 19:07:49.251908 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:49.251823    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm" event={"ID":"a70cfd95-bd84-4720-ae1a-630761e61fb1","Type":"ContainerStarted","Data":"5858631fe9c06128edf98eb170e84e080c068edc31951644d87a67d6f64fe3e8"}
Oct 30 19:07:49.251908 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:49.251877    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm" event={"ID":"a70cfd95-bd84-4720-ae1a-630761e61fb1","Type":"ContainerStarted","Data":"7bfd7a3c8c5d0a688335638c8f9e0d9e1651c1a802849d845d1223b2939ae90b"}
Oct 30 19:07:49.402266 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:49.401635    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm" podStartSLOduration=2.401595685 podStartE2EDuration="2.401595685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:49.392738672 +0000 UTC m=+1866.918613768" watchObservedRunningTime="2026-10-30 19:07:49.401595685 +0000 UTC m=+1866.927470783"
Oct 30 19:07:49.890907 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:49.890874    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:49.898090 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:49.898067    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:49.928494 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:49.928456    4544 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 19:07:49.928635 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:49.928509    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb" podUID="b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 19:07:50.280684 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:50.278216    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:50.289889 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:50.289860    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:07:50.928754 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:50.928722    4544 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 30 19:07:50.929158 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:50.928773    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb" podUID="b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 30 19:07:51.933998 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:51.933970    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:07:51.935071 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:51.935049    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:07:53.132287 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.132253    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"]
Oct 30 19:07:53.137976 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.137954    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.165487 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.165456    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-credential-socket\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.165655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.165519    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1badcf1-f26a-4a5e-a068-678c9971e21c-istiod-ca-cert\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.165655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.165568    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-workload-certs\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.165775 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.165654    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-data\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.165775 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.165686    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-token\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.165871 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.165789    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5xsw\" (UniqueName: \"kubernetes.io/projected/b1badcf1-f26a-4a5e-a068-678c9971e21c-kube-api-access-p5xsw\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.165871 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.165816    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-podinfo\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.166023 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.165974    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-envoy\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.166075 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.166047    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-workload-socket\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.192173 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.192139    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-cc079d-openshift-default-dockercfg-lndb4\""
Oct 30 19:07:53.249264 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.249228    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"]
Oct 30 19:07:53.297234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.294184    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-data\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.297234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.294237    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-token\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.297234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.294347    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5xsw\" (UniqueName: \"kubernetes.io/projected/b1badcf1-f26a-4a5e-a068-678c9971e21c-kube-api-access-p5xsw\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.297234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.294374    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-podinfo\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.297234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.294525    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-envoy\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.297234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.294617    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-workload-socket\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.297234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.294680    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-credential-socket\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.297234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.294728    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1badcf1-f26a-4a5e-a068-678c9971e21c-istiod-ca-cert\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.297234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.294775    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-workload-certs\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.297234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.295194    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-data\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.297234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.295229    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-workload-certs\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.297234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.295489    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-workload-socket\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.297234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.296019    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-credential-socket\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.297234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.296684    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1badcf1-f26a-4a5e-a068-678c9971e21c-istiod-ca-cert\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.300533 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.300320    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-podinfo\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.348301 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.348206    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-envoy\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.378271 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.378235    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-token\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.378423 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.378354    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5xsw\" (UniqueName: \"kubernetes.io/projected/b1badcf1-f26a-4a5e-a068-678c9971e21c-kube-api-access-p5xsw\") pod \"e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") " pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.454205 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.454113    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:53.900458 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:07:53.900424    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1badcf1_f26a_4a5e_a068_678c9971e21c.slice/crio-175f6d9ea51e3a93531a0460dc9ca812a59b07b248ce3cd6d73a328ed862630b WatchSource:0}: Error finding container 175f6d9ea51e3a93531a0460dc9ca812a59b07b248ce3cd6d73a328ed862630b: Status 404 returned error can't find the container with id 175f6d9ea51e3a93531a0460dc9ca812a59b07b248ce3cd6d73a328ed862630b
Oct 30 19:07:53.903332 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.903286    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:07:53.903426 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.903384    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:07:53.903472 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.903423    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:07:53.905801 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:53.905770    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"]
Oct 30 19:07:54.371850 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:54.371815    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr" event={"ID":"b1badcf1-f26a-4a5e-a068-678c9971e21c","Type":"ContainerStarted","Data":"33e8b84fc9d65eb9bbcb59ec7f1ead7c6a97f19e1594a6eb8517062a87c74171"}
Oct 30 19:07:54.372333 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:54.371882    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr" event={"ID":"b1badcf1-f26a-4a5e-a068-678c9971e21c","Type":"ContainerStarted","Data":"175f6d9ea51e3a93531a0460dc9ca812a59b07b248ce3cd6d73a328ed862630b"}
Oct 30 19:07:54.454533 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:54.454504    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:54.461431 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:54.461403    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:54.502706 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:54.502649    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr" podStartSLOduration=1.502630909 podStartE2EDuration="1.502630909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:54.499255592 +0000 UTC m=+1872.025130708" watchObservedRunningTime="2026-10-30 19:07:54.502630909 +0000 UTC m=+1872.028506001"
Oct 30 19:07:55.380305 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:55.380271    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:55.381275 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:55.381254    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:07:56.008790 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.008757    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk"]
Oct 30 19:07:56.013820 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.013795    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk"
Oct 30 19:07:56.073862 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.073831    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk"]
Oct 30 19:07:56.095023 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.094950    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-cfb433\""
Oct 30 19:07:56.095202 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.095139    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-cfb433-dockercfg-2kgbl\""
Oct 30 19:07:56.123678 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.123645    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48"]
Oct 30 19:07:56.186717 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.179873    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6frwj\" (UniqueName: \"kubernetes.io/projected/76db530e-e95b-498c-be91-e4bb9291fa43-kube-api-access-6frwj\") pod \"payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk\" (UID: \"76db530e-e95b-498c-be91-e4bb9291fa43\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk"
Oct 30 19:07:56.186717 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.180009    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76db530e-e95b-498c-be91-e4bb9291fa43-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk\" (UID: \"76db530e-e95b-498c-be91-e4bb9291fa43\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk"
Oct 30 19:07:56.203625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.194868    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48"
Oct 30 19:07:56.334134 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.334063    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6frwj\" (UniqueName: \"kubernetes.io/projected/76db530e-e95b-498c-be91-e4bb9291fa43-kube-api-access-6frwj\") pod \"payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk\" (UID: \"76db530e-e95b-498c-be91-e4bb9291fa43\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk"
Oct 30 19:07:56.348317 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.348281    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76db530e-e95b-498c-be91-e4bb9291fa43-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk\" (UID: \"76db530e-e95b-498c-be91-e4bb9291fa43\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk"
Oct 30 19:07:56.348800 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.348768    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55fmn\" (UniqueName: \"kubernetes.io/projected/50b8d8f8-30ae-4c27-9e2f-44bd646c916d-kube-api-access-55fmn\") pod \"payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48\" (UID: \"50b8d8f8-30ae-4c27-9e2f-44bd646c916d\") " pod="openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48"
Oct 30 19:07:56.349006 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.348988    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50b8d8f8-30ae-4c27-9e2f-44bd646c916d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48\" (UID: \"50b8d8f8-30ae-4c27-9e2f-44bd646c916d\") " pod="openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48"
Oct 30 19:07:56.349740 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.345269    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48"]
Oct 30 19:07:56.351280 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.351261    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76db530e-e95b-498c-be91-e4bb9291fa43-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk\" (UID: \"76db530e-e95b-498c-be91-e4bb9291fa43\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk"
Oct 30 19:07:56.376696 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.376665    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6frwj\" (UniqueName: \"kubernetes.io/projected/76db530e-e95b-498c-be91-e4bb9291fa43-kube-api-access-6frwj\") pod \"payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk\" (UID: \"76db530e-e95b-498c-be91-e4bb9291fa43\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk"
Oct 30 19:07:56.451701 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.451665    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55fmn\" (UniqueName: \"kubernetes.io/projected/50b8d8f8-30ae-4c27-9e2f-44bd646c916d-kube-api-access-55fmn\") pod \"payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48\" (UID: \"50b8d8f8-30ae-4c27-9e2f-44bd646c916d\") " pod="openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48"
Oct 30 19:07:56.452115 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.451740    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50b8d8f8-30ae-4c27-9e2f-44bd646c916d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48\" (UID: \"50b8d8f8-30ae-4c27-9e2f-44bd646c916d\") " pod="openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48"
Oct 30 19:07:56.452480 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.452459    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50b8d8f8-30ae-4c27-9e2f-44bd646c916d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48\" (UID: \"50b8d8f8-30ae-4c27-9e2f-44bd646c916d\") " pod="openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48"
Oct 30 19:07:56.511721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.511682    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55fmn\" (UniqueName: \"kubernetes.io/projected/50b8d8f8-30ae-4c27-9e2f-44bd646c916d-kube-api-access-55fmn\") pod \"payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48\" (UID: \"50b8d8f8-30ae-4c27-9e2f-44bd646c916d\") " pod="openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48"
Oct 30 19:07:56.629972 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.629891    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk"
Oct 30 19:07:56.691553 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:56.691521    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48"
Oct 30 19:07:57.047634 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:07:57.046544    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76db530e_e95b_498c_be91_e4bb9291fa43.slice/crio-a62871c33d75bcf65ca1b9ba9cce8591966ba508e68e63e2b5819bfdd326240e WatchSource:0}: Error finding container a62871c33d75bcf65ca1b9ba9cce8591966ba508e68e63e2b5819bfdd326240e: Status 404 returned error can't find the container with id a62871c33d75bcf65ca1b9ba9cce8591966ba508e68e63e2b5819bfdd326240e
Oct 30 19:07:57.086020 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:57.079809    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk"]
Oct 30 19:07:57.191673 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:07:57.190262    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50b8d8f8_30ae_4c27_9e2f_44bd646c916d.slice/crio-1a9bb383777901b29265b087b1fe6a846108e93f54d406ceed458252703da151 WatchSource:0}: Error finding container 1a9bb383777901b29265b087b1fe6a846108e93f54d406ceed458252703da151: Status 404 returned error can't find the container with id 1a9bb383777901b29265b087b1fe6a846108e93f54d406ceed458252703da151
Oct 30 19:07:57.200844 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:57.199990    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48"]
Oct 30 19:07:57.416843 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:57.416806    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48" event={"ID":"50b8d8f8-30ae-4c27-9e2f-44bd646c916d","Type":"ContainerStarted","Data":"7d6257f568a7532dda61d4b5993b9b74ac47c7f8dfe55bbfdbf3930e0bc61ec0"}
Oct 30 19:07:57.417039 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:57.416879    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48" event={"ID":"50b8d8f8-30ae-4c27-9e2f-44bd646c916d","Type":"ContainerStarted","Data":"1a9bb383777901b29265b087b1fe6a846108e93f54d406ceed458252703da151"}
Oct 30 19:07:57.417039 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:57.417011    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48"
Oct 30 19:07:57.418762 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:57.418735    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk" event={"ID":"76db530e-e95b-498c-be91-e4bb9291fa43","Type":"ContainerStarted","Data":"5165f948361c23c51dd3bb85f0d14d05b1acf695e17080dbb3a4ad984bcabd1a"}
Oct 30 19:07:57.418865 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:57.418782    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk" event={"ID":"76db530e-e95b-498c-be91-e4bb9291fa43","Type":"ContainerStarted","Data":"a62871c33d75bcf65ca1b9ba9cce8591966ba508e68e63e2b5819bfdd326240e"}
Oct 30 19:07:57.564303 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:57.564167    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48" podStartSLOduration=1.564153255 podStartE2EDuration="1.564153255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:57.560986443 +0000 UTC m=+1875.086861540" watchObservedRunningTime="2026-10-30 19:07:57.564153255 +0000 UTC m=+1875.090028351"
Oct 30 19:07:57.713925 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:57.713872    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk" podStartSLOduration=2.713858407 podStartE2EDuration="2.713858407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:57.71176503 +0000 UTC m=+1875.237640126" watchObservedRunningTime="2026-10-30 19:07:57.713858407 +0000 UTC m=+1875.239733502"
Oct 30 19:07:58.426006 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:07:58.425979    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk"
Oct 30 19:08:01.073398 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:01.063792    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:08:01.128559 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:01.128514    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:08:01.129475 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:01.129441    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:08:01.981629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:01.968715    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-j76r7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:08:01.981629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:01.968782    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7" podUID="edd86ca1-466c-4e45-8998-61d7e94fae7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:08:01.981629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:01.971474    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-zp4pg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:08:01.981629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:01.971528    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg" podUID="5ccae868-24e4-456f-ad9a-58c17d714b4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:08:02.351624 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:02.351130    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb"]
Oct 30 19:08:02.470628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:02.469678    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb"]
Oct 30 19:08:02.470628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:02.469797    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb"
Oct 30 19:08:02.535425 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:02.524831    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-cc079d\""
Oct 30 19:08:02.535425 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:02.525532    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-cc079d-dockercfg-c928t\""
Oct 30 19:08:02.564881 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:02.564838    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtxmh\" (UniqueName: \"kubernetes.io/projected/1f93fb78-4ca3-4012-8d92-9f98d8747614-kube-api-access-vtxmh\") pod \"payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb\" (UID: \"1f93fb78-4ca3-4012-8d92-9f98d8747614\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb"
Oct 30 19:08:02.565074 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:02.564915    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f93fb78-4ca3-4012-8d92-9f98d8747614-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb\" (UID: \"1f93fb78-4ca3-4012-8d92-9f98d8747614\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb"
Oct 30 19:08:02.666663 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:02.666547    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtxmh\" (UniqueName: \"kubernetes.io/projected/1f93fb78-4ca3-4012-8d92-9f98d8747614-kube-api-access-vtxmh\") pod \"payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb\" (UID: \"1f93fb78-4ca3-4012-8d92-9f98d8747614\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb"
Oct 30 19:08:02.666663 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:02.666633    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f93fb78-4ca3-4012-8d92-9f98d8747614-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb\" (UID: \"1f93fb78-4ca3-4012-8d92-9f98d8747614\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb"
Oct 30 19:08:02.667521 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:02.667498    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f93fb78-4ca3-4012-8d92-9f98d8747614-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb\" (UID: \"1f93fb78-4ca3-4012-8d92-9f98d8747614\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb"
Oct 30 19:08:02.726795 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:02.726762    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtxmh\" (UniqueName: \"kubernetes.io/projected/1f93fb78-4ca3-4012-8d92-9f98d8747614-kube-api-access-vtxmh\") pod \"payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb\" (UID: \"1f93fb78-4ca3-4012-8d92-9f98d8747614\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb"
Oct 30 19:08:02.829410 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:02.829375    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb"
Oct 30 19:08:03.232827 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:08:03.232791    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f93fb78_4ca3_4012_8d92_9f98d8747614.slice/crio-c46a09be5bcf54718bb0373e25e39f0ba03217beb536d6bc9ca19ac9ddb2d1db WatchSource:0}: Error finding container c46a09be5bcf54718bb0373e25e39f0ba03217beb536d6bc9ca19ac9ddb2d1db: Status 404 returned error can't find the container with id c46a09be5bcf54718bb0373e25e39f0ba03217beb536d6bc9ca19ac9ddb2d1db
Oct 30 19:08:03.235161 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:03.235134    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb"]
Oct 30 19:08:03.732418 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:03.732384    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb" event={"ID":"1f93fb78-4ca3-4012-8d92-9f98d8747614","Type":"ContainerStarted","Data":"105210997c89cc650c77505ab273d32969a136801d2989d9bcfb10de2946b835"}
Oct 30 19:08:03.732858 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:03.732433    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb" event={"ID":"1f93fb78-4ca3-4012-8d92-9f98d8747614","Type":"ContainerStarted","Data":"c46a09be5bcf54718bb0373e25e39f0ba03217beb536d6bc9ca19ac9ddb2d1db"}
Oct 30 19:08:03.732858 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:03.732642    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb"
Oct 30 19:08:03.851807 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:03.851752    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb" podStartSLOduration=1.851738348 podStartE2EDuration="1.851738348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:03.848066931 +0000 UTC m=+1881.373942027" watchObservedRunningTime="2026-10-30 19:08:03.851738348 +0000 UTC m=+1881.377613444"
Oct 30 19:08:05.533151 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:08:05.533110    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:08:08.429040 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:08.429011    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48"
Oct 30 19:08:09.440303 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:09.440275    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk"
Oct 30 19:08:13.371293 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:13.371259    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"]
Oct 30 19:08:13.371831 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:13.371483    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc" podUID="30db7451-7fa3-443d-bfe9-41412759e3d8" containerName="istio-proxy" containerID="cri-o://2e56fd1f367107e02e86487733010f26284de4c5d3de49f6bd3394b78308e9cb" gracePeriod=30
Oct 30 19:08:13.422150 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:13.422120    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"]
Oct 30 19:08:13.422346 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:13.422324    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g" podUID="90f8bdc0-d655-4c71-a348-9fa661ece9c6" containerName="istio-proxy" containerID="cri-o://f739ceadc5a512377ca2b7119646e928233e7244717e5e3eb6d71b8588bf7864" gracePeriod=30
Oct 30 19:08:13.427526 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:13.427498    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"]
Oct 30 19:08:13.427760 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:13.427736    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75" podUID="be77624e-eeb7-44f4-8d9e-dfd58897e555" containerName="istio-proxy" containerID="cri-o://746ebc4f7c72e36da5fccd7344bb932af66b8a5f37245b10f5aaaca1927d188f" gracePeriod=30
Oct 30 19:08:13.473218 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:13.473180    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"]
Oct 30 19:08:13.473472 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:13.473425    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6" podUID="60c1bed3-9d33-4979-815a-84a7acc09c38" containerName="istio-proxy" containerID="cri-o://85574b987656d051cc5ebdcfdbec662c1baa5778a38a7d9280f1c2c3ecfb0dc6" gracePeriod=30
Oct 30 19:08:14.743070 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:14.743038    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb"
Oct 30 19:08:15.963153 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:15.962809    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:08:15.963579 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:15.963207    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75" podUID="be77624e-eeb7-44f4-8d9e-dfd58897e555" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:08:18.577084 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.577051    4544 patch_prober.go:29] interesting pod/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 19:08:18.577426 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.577127    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc" podUID="30db7451-7fa3-443d-bfe9-41412759e3d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 19:08:18.756127 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.756098    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 19:08:18.821024 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.820994    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 19:08:18.854685 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.854646    4544 generic.go:356] "Generic (PLEG): container finished" podID="60c1bed3-9d33-4979-815a-84a7acc09c38" containerID="85574b987656d051cc5ebdcfdbec662c1baa5778a38a7d9280f1c2c3ecfb0dc6" exitCode=0
Oct 30 19:08:18.854865 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.854796    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6" event={"ID":"60c1bed3-9d33-4979-815a-84a7acc09c38","Type":"ContainerDied","Data":"85574b987656d051cc5ebdcfdbec662c1baa5778a38a7d9280f1c2c3ecfb0dc6"}
Oct 30 19:08:18.856524 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.856496    4544 generic.go:356] "Generic (PLEG): container finished" podID="30db7451-7fa3-443d-bfe9-41412759e3d8" containerID="2e56fd1f367107e02e86487733010f26284de4c5d3de49f6bd3394b78308e9cb" exitCode=0
Oct 30 19:08:18.856670 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.856647    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc" event={"ID":"30db7451-7fa3-443d-bfe9-41412759e3d8","Type":"ContainerDied","Data":"2e56fd1f367107e02e86487733010f26284de4c5d3de49f6bd3394b78308e9cb"}
Oct 30 19:08:18.856749 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.856688    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc" event={"ID":"30db7451-7fa3-443d-bfe9-41412759e3d8","Type":"ContainerDied","Data":"5e3c7418608bfcad6946cfed32b90be0f6462e37ac85f9f94c9d2c738f479f0e"}
Oct 30 19:08:18.856749 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.856718    4544 scope.go:122] "RemoveContainer" containerID="2e56fd1f367107e02e86487733010f26284de4c5d3de49f6bd3394b78308e9cb"
Oct 30 19:08:18.856875 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.856860    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"
Oct 30 19:08:18.862136 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.862098    4544 generic.go:356] "Generic (PLEG): container finished" podID="be77624e-eeb7-44f4-8d9e-dfd58897e555" containerID="746ebc4f7c72e36da5fccd7344bb932af66b8a5f37245b10f5aaaca1927d188f" exitCode=0
Oct 30 19:08:18.862309 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.862293    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75" event={"ID":"be77624e-eeb7-44f4-8d9e-dfd58897e555","Type":"ContainerDied","Data":"746ebc4f7c72e36da5fccd7344bb932af66b8a5f37245b10f5aaaca1927d188f"}
Oct 30 19:08:18.864491 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.864464    4544 generic.go:356] "Generic (PLEG): container finished" podID="90f8bdc0-d655-4c71-a348-9fa661ece9c6" containerID="f739ceadc5a512377ca2b7119646e928233e7244717e5e3eb6d71b8588bf7864" exitCode=0
Oct 30 19:08:18.864689 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.864562    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"
Oct 30 19:08:18.864689 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.864652    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g" event={"ID":"90f8bdc0-d655-4c71-a348-9fa661ece9c6","Type":"ContainerDied","Data":"f739ceadc5a512377ca2b7119646e928233e7244717e5e3eb6d71b8588bf7864"}
Oct 30 19:08:18.864856 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.864692    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g" event={"ID":"90f8bdc0-d655-4c71-a348-9fa661ece9c6","Type":"ContainerDied","Data":"ccf91ba3f0e3755641a54a6ee8176622cd2442b15e90fd3b47dc1d7dd94f7c35"}
Oct 30 19:08:18.881024 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.881000    4544 scope.go:122] "RemoveContainer" containerID="2e56fd1f367107e02e86487733010f26284de4c5d3de49f6bd3394b78308e9cb"
Oct 30 19:08:18.881980 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:08:18.881955    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e56fd1f367107e02e86487733010f26284de4c5d3de49f6bd3394b78308e9cb\": container with ID starting with 2e56fd1f367107e02e86487733010f26284de4c5d3de49f6bd3394b78308e9cb not found: ID does not exist" containerID="2e56fd1f367107e02e86487733010f26284de4c5d3de49f6bd3394b78308e9cb"
Oct 30 19:08:18.882080 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.881990    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e56fd1f367107e02e86487733010f26284de4c5d3de49f6bd3394b78308e9cb"} err="failed to get container status \"2e56fd1f367107e02e86487733010f26284de4c5d3de49f6bd3394b78308e9cb\": rpc error: code = NotFound desc = could not find container \"2e56fd1f367107e02e86487733010f26284de4c5d3de49f6bd3394b78308e9cb\": container with ID starting with 2e56fd1f367107e02e86487733010f26284de4c5d3de49f6bd3394b78308e9cb not found: ID does not exist"
Oct 30 19:08:18.882080 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.882012    4544 scope.go:122] "RemoveContainer" containerID="888e1cc8d1cc67f4c3a25002746cd0b478cd3c808ace5d51a6344c4dc8fb8149"
Oct 30 19:08:18.883161 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.883118    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-podinfo\") pod \"30db7451-7fa3-443d-bfe9-41412759e3d8\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") "
Oct 30 19:08:18.883258 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.883178    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-credential-socket\") pod \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") "
Oct 30 19:08:18.883384 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.883329    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-workload-certs\") pod \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") "
Oct 30 19:08:18.883384 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.883376    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-workload-certs\") pod \"30db7451-7fa3-443d-bfe9-41412759e3d8\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") "
Oct 30 19:08:18.883494 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.883412    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90f8bdc0-d655-4c71-a348-9fa661ece9c6-kube-api-access-m7tj6\" (UniqueName: \"kubernetes.io/projected/90f8bdc0-d655-4c71-a348-9fa661ece9c6-kube-api-access-m7tj6\") pod \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") "
Oct 30 19:08:18.883494 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.883442    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istiod-ca-cert\") pod \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") "
Oct 30 19:08:18.883624 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.883504    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-envoy\") pod \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") "
Oct 30 19:08:18.883624 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.883542    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-credential-socket\") pod \"30db7451-7fa3-443d-bfe9-41412759e3d8\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") "
Oct 30 19:08:18.883624 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.883589    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30db7451-7fa3-443d-bfe9-41412759e3d8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30db7451-7fa3-443d-bfe9-41412759e3d8-istiod-ca-cert\") pod \"30db7451-7fa3-443d-bfe9-41412759e3d8\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") "
Oct 30 19:08:18.883783 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.883684    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-workload-socket\") pod \"30db7451-7fa3-443d-bfe9-41412759e3d8\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") "
Oct 30 19:08:18.883783 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.883767    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-data\") pod \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") "
Oct 30 19:08:18.883887 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.883833    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-token\" (UniqueName: \"kubernetes.io/projected/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-token\") pod \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") "
Oct 30 19:08:18.883887 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.883850    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-workload-certs" pod "90f8bdc0-d655-4c71-a348-9fa661ece9c6" (UID: "90f8bdc0-d655-4c71-a348-9fa661ece9c6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.884218 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.884192    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-workload-certs" pod "30db7451-7fa3-443d-bfe9-41412759e3d8" (UID: "30db7451-7fa3-443d-bfe9-41412759e3d8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.884218 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.884208    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-credential-socket" pod "30db7451-7fa3-443d-bfe9-41412759e3d8" (UID: "30db7451-7fa3-443d-bfe9-41412759e3d8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.884374 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.884349    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-workload-socket" pod "30db7451-7fa3-443d-bfe9-41412759e3d8" (UID: "30db7451-7fa3-443d-bfe9-41412759e3d8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.884481 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.884462    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istiod-ca-cert" pod "90f8bdc0-d655-4c71-a348-9fa661ece9c6" (UID: "90f8bdc0-d655-4c71-a348-9fa661ece9c6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:08:18.884560 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.884512    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-token\" (UniqueName: \"kubernetes.io/projected/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-token\") pod \"30db7451-7fa3-443d-bfe9-41412759e3d8\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") "
Oct 30 19:08:18.884648 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.884631    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-data\") pod \"30db7451-7fa3-443d-bfe9-41412759e3d8\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") "
Oct 30 19:08:18.884706 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.884666    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-workload-socket\") pod \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") "
Oct 30 19:08:18.884763 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.884715    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30db7451-7fa3-443d-bfe9-41412759e3d8-kube-api-access-4ccbm\" (UniqueName: \"kubernetes.io/projected/30db7451-7fa3-443d-bfe9-41412759e3d8-kube-api-access-4ccbm\") pod \"30db7451-7fa3-443d-bfe9-41412759e3d8\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") "
Oct 30 19:08:18.884816 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.884763    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-envoy\") pod \"30db7451-7fa3-443d-bfe9-41412759e3d8\" (UID: \"30db7451-7fa3-443d-bfe9-41412759e3d8\") "
Oct 30 19:08:18.884868 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.884812    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-podinfo\") pod \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\" (UID: \"90f8bdc0-d655-4c71-a348-9fa661ece9c6\") "
Oct 30 19:08:18.885531 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.884543    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30db7451-7fa3-443d-bfe9-41412759e3d8-istiod-ca-cert" pod "30db7451-7fa3-443d-bfe9-41412759e3d8" (UID: "30db7451-7fa3-443d-bfe9-41412759e3d8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:08:18.885531 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.884567    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-data" pod "90f8bdc0-d655-4c71-a348-9fa661ece9c6" (UID: "90f8bdc0-d655-4c71-a348-9fa661ece9c6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.885531 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.884829    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-data" pod "30db7451-7fa3-443d-bfe9-41412759e3d8" (UID: "30db7451-7fa3-443d-bfe9-41412759e3d8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.885531 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.884987    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-workload-socket" pod "90f8bdc0-d655-4c71-a348-9fa661ece9c6" (UID: "90f8bdc0-d655-4c71-a348-9fa661ece9c6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.885531 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.885099    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-credential-socket" pod "90f8bdc0-d655-4c71-a348-9fa661ece9c6" (UID: "90f8bdc0-d655-4c71-a348-9fa661ece9c6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.885926 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.885637    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.885926 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.885658    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.885926 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.885673    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.885926 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.885688    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.885926 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.885703    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.885926 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.885717    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.885926 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.885732    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.885926 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.885747    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.885926 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.885763    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30db7451-7fa3-443d-bfe9-41412759e3d8-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.885926 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.885777    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.889241 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.889214    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-token" pod "30db7451-7fa3-443d-bfe9-41412759e3d8" (UID: "30db7451-7fa3-443d-bfe9-41412759e3d8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:18.891941 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.891914    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-envoy" pod "30db7451-7fa3-443d-bfe9-41412759e3d8" (UID: "30db7451-7fa3-443d-bfe9-41412759e3d8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.892032 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.891991    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-podinfo" pod "30db7451-7fa3-443d-bfe9-41412759e3d8" (UID: "30db7451-7fa3-443d-bfe9-41412759e3d8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:08:18.892118 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.892094    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-podinfo" pod "90f8bdc0-d655-4c71-a348-9fa661ece9c6" (UID: "90f8bdc0-d655-4c71-a348-9fa661ece9c6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:08:18.892284 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.892265    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-envoy" pod "90f8bdc0-d655-4c71-a348-9fa661ece9c6" (UID: "90f8bdc0-d655-4c71-a348-9fa661ece9c6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.893101 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.893066    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-token" pod "90f8bdc0-d655-4c71-a348-9fa661ece9c6" (UID: "90f8bdc0-d655-4c71-a348-9fa661ece9c6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:18.895435 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.895110    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30db7451-7fa3-443d-bfe9-41412759e3d8-kube-api-access-4ccbm" pod "30db7451-7fa3-443d-bfe9-41412759e3d8" (UID: "30db7451-7fa3-443d-bfe9-41412759e3d8"). InnerVolumeSpecName "kube-api-access-4ccbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:18.895435 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.895243    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 19:08:18.899515 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.899480    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 19:08:18.901995 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.901906    4544 scope.go:122] "RemoveContainer" containerID="f739ceadc5a512377ca2b7119646e928233e7244717e5e3eb6d71b8588bf7864"
Oct 30 19:08:18.903221 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.903197    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90f8bdc0-d655-4c71-a348-9fa661ece9c6-kube-api-access-m7tj6" pod "90f8bdc0-d655-4c71-a348-9fa661ece9c6" (UID: "90f8bdc0-d655-4c71-a348-9fa661ece9c6"). InnerVolumeSpecName "kube-api-access-m7tj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:18.920552 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.920530    4544 scope.go:122] "RemoveContainer" containerID="f739ceadc5a512377ca2b7119646e928233e7244717e5e3eb6d71b8588bf7864"
Oct 30 19:08:18.920929 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:08:18.920864    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f739ceadc5a512377ca2b7119646e928233e7244717e5e3eb6d71b8588bf7864\": container with ID starting with f739ceadc5a512377ca2b7119646e928233e7244717e5e3eb6d71b8588bf7864 not found: ID does not exist" containerID="f739ceadc5a512377ca2b7119646e928233e7244717e5e3eb6d71b8588bf7864"
Oct 30 19:08:18.920929 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.920901    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f739ceadc5a512377ca2b7119646e928233e7244717e5e3eb6d71b8588bf7864"} err="failed to get container status \"f739ceadc5a512377ca2b7119646e928233e7244717e5e3eb6d71b8588bf7864\": rpc error: code = NotFound desc = could not find container \"f739ceadc5a512377ca2b7119646e928233e7244717e5e3eb6d71b8588bf7864\": container with ID starting with f739ceadc5a512377ca2b7119646e928233e7244717e5e3eb6d71b8588bf7864 not found: ID does not exist"
Oct 30 19:08:18.987318 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.987229    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-podinfo\") pod \"60c1bed3-9d33-4979-815a-84a7acc09c38\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") "
Oct 30 19:08:18.987318 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.987286    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-workload-socket\") pod \"be77624e-eeb7-44f4-8d9e-dfd58897e555\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") "
Oct 30 19:08:18.987554 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.987379    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-credential-socket\") pod \"60c1bed3-9d33-4979-815a-84a7acc09c38\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") "
Oct 30 19:08:18.987554 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.987441    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-workload-certs\") pod \"be77624e-eeb7-44f4-8d9e-dfd58897e555\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") "
Oct 30 19:08:18.987554 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.987472    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-workload-certs\") pod \"60c1bed3-9d33-4979-815a-84a7acc09c38\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") "
Oct 30 19:08:18.987554 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.987520    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-envoy\") pod \"60c1bed3-9d33-4979-815a-84a7acc09c38\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") "
Oct 30 19:08:18.988435 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.987586    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-workload-socket" pod "be77624e-eeb7-44f4-8d9e-dfd58897e555" (UID: "be77624e-eeb7-44f4-8d9e-dfd58897e555"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.988435 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.988281    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-workload-socket\") pod \"60c1bed3-9d33-4979-815a-84a7acc09c38\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") "
Oct 30 19:08:18.988435 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.988326    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-token\" (UniqueName: \"kubernetes.io/projected/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-token\") pod \"be77624e-eeb7-44f4-8d9e-dfd58897e555\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") "
Oct 30 19:08:18.988435 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.988259    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-credential-socket" pod "60c1bed3-9d33-4979-815a-84a7acc09c38" (UID: "60c1bed3-9d33-4979-815a-84a7acc09c38"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.988435 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.988297    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-workload-certs" pod "60c1bed3-9d33-4979-815a-84a7acc09c38" (UID: "60c1bed3-9d33-4979-815a-84a7acc09c38"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.988435 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.988412    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-token\" (UniqueName: \"kubernetes.io/projected/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-token\") pod \"60c1bed3-9d33-4979-815a-84a7acc09c38\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") "
Oct 30 19:08:18.988841 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.988479    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-data\") pod \"be77624e-eeb7-44f4-8d9e-dfd58897e555\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") "
Oct 30 19:08:18.988841 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.988593    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-envoy\") pod \"be77624e-eeb7-44f4-8d9e-dfd58897e555\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") "
Oct 30 19:08:18.988841 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.988694    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be77624e-eeb7-44f4-8d9e-dfd58897e555-kube-api-access-s8gq7\" (UniqueName: \"kubernetes.io/projected/be77624e-eeb7-44f4-8d9e-dfd58897e555-kube-api-access-s8gq7\") pod \"be77624e-eeb7-44f4-8d9e-dfd58897e555\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") "
Oct 30 19:08:18.988841 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.988734    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-credential-socket\") pod \"be77624e-eeb7-44f4-8d9e-dfd58897e555\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") "
Oct 30 19:08:18.988841 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.988817    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-podinfo\") pod \"be77624e-eeb7-44f4-8d9e-dfd58897e555\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") "
Oct 30 19:08:18.989096 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.988865    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/be77624e-eeb7-44f4-8d9e-dfd58897e555-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be77624e-eeb7-44f4-8d9e-dfd58897e555-istiod-ca-cert\") pod \"be77624e-eeb7-44f4-8d9e-dfd58897e555\" (UID: \"be77624e-eeb7-44f4-8d9e-dfd58897e555\") "
Oct 30 19:08:18.989096 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.988897    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60c1bed3-9d33-4979-815a-84a7acc09c38-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60c1bed3-9d33-4979-815a-84a7acc09c38-istiod-ca-cert\") pod \"60c1bed3-9d33-4979-815a-84a7acc09c38\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") "
Oct 30 19:08:18.989096 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.988927    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-data\") pod \"60c1bed3-9d33-4979-815a-84a7acc09c38\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") "
Oct 30 19:08:18.989096 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.988967    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60c1bed3-9d33-4979-815a-84a7acc09c38-kube-api-access-ndwvg\" (UniqueName: \"kubernetes.io/projected/60c1bed3-9d33-4979-815a-84a7acc09c38-kube-api-access-ndwvg\") pod \"60c1bed3-9d33-4979-815a-84a7acc09c38\" (UID: \"60c1bed3-9d33-4979-815a-84a7acc09c38\") "
Oct 30 19:08:18.990567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.989350    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-workload-socket" pod "60c1bed3-9d33-4979-815a-84a7acc09c38" (UID: "60c1bed3-9d33-4979-815a-84a7acc09c38"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.990567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.989795    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.990567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.989816    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7tj6\" (UniqueName: \"kubernetes.io/projected/90f8bdc0-d655-4c71-a348-9fa661ece9c6-kube-api-access-m7tj6\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.990567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.989832    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.990567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.989846    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.990567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.989860    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.990567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.989875    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.990567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.989888    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.990567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.989903    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.990567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.989918    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.990567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.989932    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ccbm\" (UniqueName: \"kubernetes.io/projected/30db7451-7fa3-443d-bfe9-41412759e3d8-kube-api-access-4ccbm\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.990567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.989947    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30db7451-7fa3-443d-bfe9-41412759e3d8-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.990567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.989960    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90f8bdc0-d655-4c71-a348-9fa661ece9c6-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:18.990567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.990181    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-workload-certs" pod "be77624e-eeb7-44f4-8d9e-dfd58897e555" (UID: "be77624e-eeb7-44f4-8d9e-dfd58897e555"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.990567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.990395    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-data" pod "be77624e-eeb7-44f4-8d9e-dfd58897e555" (UID: "be77624e-eeb7-44f4-8d9e-dfd58897e555"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.991494 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.991128    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60c1bed3-9d33-4979-815a-84a7acc09c38-istiod-ca-cert" pod "60c1bed3-9d33-4979-815a-84a7acc09c38" (UID: "60c1bed3-9d33-4979-815a-84a7acc09c38"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:08:18.991494 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.991380    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-data" pod "60c1bed3-9d33-4979-815a-84a7acc09c38" (UID: "60c1bed3-9d33-4979-815a-84a7acc09c38"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.991494 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.991379    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be77624e-eeb7-44f4-8d9e-dfd58897e555-istiod-ca-cert" pod "be77624e-eeb7-44f4-8d9e-dfd58897e555" (UID: "be77624e-eeb7-44f4-8d9e-dfd58897e555"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:08:18.991912 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.991875    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-credential-socket" pod "be77624e-eeb7-44f4-8d9e-dfd58897e555" (UID: "be77624e-eeb7-44f4-8d9e-dfd58897e555"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.992119 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.992061    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-envoy" pod "60c1bed3-9d33-4979-815a-84a7acc09c38" (UID: "60c1bed3-9d33-4979-815a-84a7acc09c38"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.992119 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.992079    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-podinfo" pod "60c1bed3-9d33-4979-815a-84a7acc09c38" (UID: "60c1bed3-9d33-4979-815a-84a7acc09c38"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:08:18.992767 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.992742    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-token" pod "60c1bed3-9d33-4979-815a-84a7acc09c38" (UID: "60c1bed3-9d33-4979-815a-84a7acc09c38"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:18.992870 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.992775    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-token" pod "be77624e-eeb7-44f4-8d9e-dfd58897e555" (UID: "be77624e-eeb7-44f4-8d9e-dfd58897e555"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:18.994052 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.994027    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-envoy" pod "be77624e-eeb7-44f4-8d9e-dfd58897e555" (UID: "be77624e-eeb7-44f4-8d9e-dfd58897e555"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:18.994144 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.994065    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60c1bed3-9d33-4979-815a-84a7acc09c38-kube-api-access-ndwvg" pod "60c1bed3-9d33-4979-815a-84a7acc09c38" (UID: "60c1bed3-9d33-4979-815a-84a7acc09c38"). InnerVolumeSpecName "kube-api-access-ndwvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:18.995638 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.995616    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be77624e-eeb7-44f4-8d9e-dfd58897e555-kube-api-access-s8gq7" pod "be77624e-eeb7-44f4-8d9e-dfd58897e555" (UID: "be77624e-eeb7-44f4-8d9e-dfd58897e555"). InnerVolumeSpecName "kube-api-access-s8gq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:18.995895 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:18.995875    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-podinfo" pod "be77624e-eeb7-44f4-8d9e-dfd58897e555" (UID: "be77624e-eeb7-44f4-8d9e-dfd58897e555"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:08:19.096364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.092950    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:19.096364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.092983    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be77624e-eeb7-44f4-8d9e-dfd58897e555-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:19.096364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.092998    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60c1bed3-9d33-4979-815a-84a7acc09c38-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:19.096364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.093012    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:19.096364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.093026    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndwvg\" (UniqueName: \"kubernetes.io/projected/60c1bed3-9d33-4979-815a-84a7acc09c38-kube-api-access-ndwvg\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:19.096364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.093042    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:19.096364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.093056    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:19.096364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.093071    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:19.096364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.093084    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:19.096364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.093098    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60c1bed3-9d33-4979-815a-84a7acc09c38-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:19.096364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.093113    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:19.096364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.093128    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:19.096364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.093145    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8gq7\" (UniqueName: \"kubernetes.io/projected/be77624e-eeb7-44f4-8d9e-dfd58897e555-kube-api-access-s8gq7\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:19.096364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.093158    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be77624e-eeb7-44f4-8d9e-dfd58897e555-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:19.546003 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.545966    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"]
Oct 30 19:08:19.568651 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.560466    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-t67wc"]
Oct 30 19:08:19.672619 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.672515    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"]
Oct 30 19:08:19.675882 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.675854    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-r9w8g"]
Oct 30 19:08:19.873109 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.873081    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"
Oct 30 19:08:19.873109 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.873097    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75" event={"ID":"be77624e-eeb7-44f4-8d9e-dfd58897e555","Type":"ContainerDied","Data":"1545326c5c92188d6119166f704fc3e50db15dd2572720ac29944a024a72f64e"}
Oct 30 19:08:19.873350 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.873155    4544 scope.go:122] "RemoveContainer" containerID="746ebc4f7c72e36da5fccd7344bb932af66b8a5f37245b10f5aaaca1927d188f"
Oct 30 19:08:19.876178 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.876156    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"
Oct 30 19:08:19.876386 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.876199    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6" event={"ID":"60c1bed3-9d33-4979-815a-84a7acc09c38","Type":"ContainerDied","Data":"a8ed1051d7a1bc34db123bd2552b4ebd68e5f62136693a192a96efc9983edf77"}
Oct 30 19:08:19.886592 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:19.886572    4544 scope.go:122] "RemoveContainer" containerID="85574b987656d051cc5ebdcfdbec662c1baa5778a38a7d9280f1c2c3ecfb0dc6"
Oct 30 19:08:20.118712 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:20.118680    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"]
Oct 30 19:08:20.126544 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:20.126514    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-mhsm6"]
Oct 30 19:08:20.236783 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:20.236752    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"]
Oct 30 19:08:20.244501 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:20.244478    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-75e91b-openshift-default-5546f5b579-zdp75"]
Oct 30 19:08:20.606210 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:20.606178    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30db7451-7fa3-443d-bfe9-41412759e3d8" path="/var/lib/kubelet/pods/30db7451-7fa3-443d-bfe9-41412759e3d8/volumes"
Oct 30 19:08:20.606683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:20.606669    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60c1bed3-9d33-4979-815a-84a7acc09c38" path="/var/lib/kubelet/pods/60c1bed3-9d33-4979-815a-84a7acc09c38/volumes"
Oct 30 19:08:20.607344 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:20.607328    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="90f8bdc0-d655-4c71-a348-9fa661ece9c6" path="/var/lib/kubelet/pods/90f8bdc0-d655-4c71-a348-9fa661ece9c6/volumes"
Oct 30 19:08:20.607798 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:20.607784    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be77624e-eeb7-44f4-8d9e-dfd58897e555" path="/var/lib/kubelet/pods/be77624e-eeb7-44f4-8d9e-dfd58897e555/volumes"
Oct 30 19:08:27.312358 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.312317    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"]
Oct 30 19:08:27.313039 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.313019    4544 cpu_manager.go:414] "containerMap: removing container" podUID="60c1bed3-9d33-4979-815a-84a7acc09c38" containerName="istio-proxy"
Oct 30 19:08:27.313039 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.313038    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60c1bed3-9d33-4979-815a-84a7acc09c38" containerName="istio-proxy"
Oct 30 19:08:27.313157 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.313067    4544 cpu_manager.go:414] "containerMap: removing container" podUID="be77624e-eeb7-44f4-8d9e-dfd58897e555" containerName="istio-proxy"
Oct 30 19:08:27.313157 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.313072    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be77624e-eeb7-44f4-8d9e-dfd58897e555" containerName="istio-proxy"
Oct 30 19:08:27.313157 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.313089    4544 cpu_manager.go:414] "containerMap: removing container" podUID="30db7451-7fa3-443d-bfe9-41412759e3d8" containerName="istio-proxy"
Oct 30 19:08:27.313157 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.313094    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30db7451-7fa3-443d-bfe9-41412759e3d8" containerName="istio-proxy"
Oct 30 19:08:27.313157 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.313102    4544 cpu_manager.go:414] "containerMap: removing container" podUID="be77624e-eeb7-44f4-8d9e-dfd58897e555" containerName="istio-proxy"
Oct 30 19:08:27.313157 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.313108    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be77624e-eeb7-44f4-8d9e-dfd58897e555" containerName="istio-proxy"
Oct 30 19:08:27.313157 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.313124    4544 cpu_manager.go:414] "containerMap: removing container" podUID="90f8bdc0-d655-4c71-a348-9fa661ece9c6" containerName="istio-proxy"
Oct 30 19:08:27.313157 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.313128    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90f8bdc0-d655-4c71-a348-9fa661ece9c6" containerName="istio-proxy"
Oct 30 19:08:27.313554 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.313230    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="90f8bdc0-d655-4c71-a348-9fa661ece9c6" containerName="istio-proxy"
Oct 30 19:08:27.313554 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.313241    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="60c1bed3-9d33-4979-815a-84a7acc09c38" containerName="istio-proxy"
Oct 30 19:08:27.313554 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.313250    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="be77624e-eeb7-44f4-8d9e-dfd58897e555" containerName="istio-proxy"
Oct 30 19:08:27.313554 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.313258    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="30db7451-7fa3-443d-bfe9-41412759e3d8" containerName="istio-proxy"
Oct 30 19:08:27.318861 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.318835    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:08:27.342832 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.342792    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-maas-api-tls\") pod \"maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc\" (UID: \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:08:27.343065 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.343037    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ps8d\" (UniqueName: \"kubernetes.io/projected/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-kube-api-access-5ps8d\") pod \"maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc\" (UID: \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:08:27.343176 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.343156    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-metrics-certs\") pod \"maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc\" (UID: \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:08:27.343269 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.343252    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-openshift-service-ca\") pod \"maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc\" (UID: \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:08:27.399468 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.399432    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"]
Oct 30 19:08:27.444587 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.444550    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ps8d\" (UniqueName: \"kubernetes.io/projected/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-kube-api-access-5ps8d\") pod \"maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc\" (UID: \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:08:27.444786 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.444645    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-metrics-certs\") pod \"maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc\" (UID: \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:08:27.444786 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.444728    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-openshift-service-ca\") pod \"maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc\" (UID: \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:08:27.444922 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.444787    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-maas-api-tls\") pod \"maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc\" (UID: \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:08:27.445462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.445436    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-openshift-service-ca\") pod \"maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc\" (UID: \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:08:27.447317 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.447286    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-metrics-certs\") pod \"maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc\" (UID: \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:08:27.451321 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.451301    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-2ef49a-serving-cert\""
Oct 30 19:08:27.455510 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:08:27.455490    4544 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-serving-cert: secret "maas-api-e2e-api-a-2ef49a-serving-cert" not found
Oct 30 19:08:27.455646 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:08:27.455597    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-maas-api-tls podName:ce2fa48e-5bbf-4846-8068-2c8532e3e37e nodeName:}" failed. No retries permitted until 2026-10-30 19:08:27.955576818 +0000 UTC m=+1905.481451910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-maas-api-tls") pod "maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc" (UID: "ce2fa48e-5bbf-4846-8068-2c8532e3e37e") : secret "maas-api-e2e-api-a-2ef49a-serving-cert" not found
Oct 30 19:08:27.505279 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:27.505244    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ps8d\" (UniqueName: \"kubernetes.io/projected/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-kube-api-access-5ps8d\") pod \"maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc\" (UID: \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:08:28.054245 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:28.054214    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-maas-api-tls\") pod \"maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc\" (UID: \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:08:28.060627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:28.056802    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-maas-api-tls\") pod \"maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc\" (UID: \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:08:28.234848 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:28.234819    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:08:28.653859 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:08:28.653822    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce2fa48e_5bbf_4846_8068_2c8532e3e37e.slice/crio-f2bfada5ad20d3cd9456532e4f074a132ea49bac5f7e56cfe59494e275b2188f WatchSource:0}: Error finding container f2bfada5ad20d3cd9456532e4f074a132ea49bac5f7e56cfe59494e275b2188f: Status 404 returned error can't find the container with id f2bfada5ad20d3cd9456532e4f074a132ea49bac5f7e56cfe59494e275b2188f
Oct 30 19:08:28.655636 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:28.655594    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"]
Oct 30 19:08:28.952938 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:28.952850    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc" event={"ID":"ce2fa48e-5bbf-4846-8068-2c8532e3e37e","Type":"ContainerStarted","Data":"f2bfada5ad20d3cd9456532e4f074a132ea49bac5f7e56cfe59494e275b2188f"}
Oct 30 19:08:29.963838 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:29.963734    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc" event={"ID":"ce2fa48e-5bbf-4846-8068-2c8532e3e37e","Type":"ContainerStarted","Data":"36af21b9b51d31b5f4064b335a6e2ccc14b33d530f54afeaceef7e4649c5df76"}
Oct 30 19:08:29.963838 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:29.963827    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:08:30.092370 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:30.092315    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc" podStartSLOduration=2.16796906 podStartE2EDuration="3.092299661s" totalImagesPullingTime="924.330601ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:30.087203938 +0000 UTC m=+1907.613079036" watchObservedRunningTime="2026-10-30 19:08:30.092299661 +0000 UTC m=+1907.618174802"
Oct 30 19:08:32.577814 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:32.577773    4544 patch_prober.go:29] interesting pod/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:08:32.578224 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:32.577836    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2" podUID="fe43caaf-eb11-4b75-a185-66ab33940e27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 19:08:32.993506 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:32.993458    4544 generic.go:356] "Generic (PLEG): container finished" podID="fe43caaf-eb11-4b75-a185-66ab33940e27" containerID="46891d3a4bed07102f87a20dde1eab6e80588d4a99be77e94bcd8b1fdc6cf09e" exitCode=0
Oct 30 19:08:32.993756 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:32.993596    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2" event={"ID":"fe43caaf-eb11-4b75-a185-66ab33940e27","Type":"ContainerDied","Data":"46891d3a4bed07102f87a20dde1eab6e80588d4a99be77e94bcd8b1fdc6cf09e"}
Oct 30 19:08:32.994145 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:32.994123    4544 scope.go:122] "RemoveContainer" containerID="46891d3a4bed07102f87a20dde1eab6e80588d4a99be77e94bcd8b1fdc6cf09e"
Oct 30 19:08:32.996361 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:32.996317    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:08:32.996565 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:32.996539    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:08:32.996698 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:32.996588    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:08:34.002411 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:34.002374    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2" event={"ID":"fe43caaf-eb11-4b75-a185-66ab33940e27","Type":"ContainerStarted","Data":"4f39785031d4b8611956b238a42fc8ec7a06862568d6a88191ca837cdfc1333b"}
Oct 30 19:08:34.130354 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:34.130311    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:08:34.130530 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:34.130454    4544 patch_prober.go:29] interesting pod/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:08:34.130530 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:34.130502    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2" podUID="fe43caaf-eb11-4b75-a185-66ab33940e27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 19:08:35.130041 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:35.130010    4544 patch_prober.go:29] interesting pod/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:08:35.130512 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:35.130066    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2" podUID="fe43caaf-eb11-4b75-a185-66ab33940e27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 19:08:35.531234 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:08:35.531186    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:08:35.988129 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:35.988053    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:08:36.135386 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:36.133884    4544 patch_prober.go:29] interesting pod/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:08:36.135386 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:36.133937    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2" podUID="fe43caaf-eb11-4b75-a185-66ab33940e27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 19:08:37.130209 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.130174    4544 patch_prober.go:29] interesting pod/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:08:37.130400 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.130229    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2" podUID="fe43caaf-eb11-4b75-a185-66ab33940e27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 19:08:37.815253 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.815220    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"]
Oct 30 19:08:37.816052 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.816036    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="be77624e-eeb7-44f4-8d9e-dfd58897e555" containerName="istio-proxy"
Oct 30 19:08:37.819848 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.819831    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.847294 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.847268    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-podinfo\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.847417 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.847306    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-workload-certs\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.847417 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.847338    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-token\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.847417 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.847414    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-credential-socket\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.847525 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.847455    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlpdt\" (UniqueName: \"kubernetes.io/projected/0456a5cd-81b2-4dd1-9e36-a51808065ad0-kube-api-access-qlpdt\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.847525 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.847470    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-data\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.847525 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.847491    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-envoy\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.847639 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.847524    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istiod-ca-cert\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.847639 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.847565    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-workload-socket\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.880969 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.880937    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-071cf0-openshift-default-dockercfg-jfzgt\""
Oct 30 19:08:37.934693 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.934659    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"]
Oct 30 19:08:37.949128 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.949096    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-credential-socket\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.949331 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.949152    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlpdt\" (UniqueName: \"kubernetes.io/projected/0456a5cd-81b2-4dd1-9e36-a51808065ad0-kube-api-access-qlpdt\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.949331 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.949173    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-data\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.949331 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.949198    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-envoy\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.949331 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.949229    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istiod-ca-cert\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.949331 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.949270    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-workload-socket\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.949632 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.949389    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-podinfo\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.949632 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.949429    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-workload-certs\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.949632 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.949480    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-token\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.950069 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.950044    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istiod-ca-cert\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.950400 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.950353    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-data\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.950526 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.950402    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-workload-socket\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.950526 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.950429    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-credential-socket\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.950526 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.950459    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-workload-certs\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.951942 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.951923    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-podinfo\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:37.952732 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:37.952714    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-envoy\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:38.008288 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:38.008240    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlpdt\" (UniqueName: \"kubernetes.io/projected/0456a5cd-81b2-4dd1-9e36-a51808065ad0-kube-api-access-qlpdt\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:38.008484 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:38.008459    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-token\") pod \"e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") " pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:38.137676 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:38.134059    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:38.140621 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:38.138331    4544 patch_prober.go:29] interesting pod/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:08:38.140621 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:38.138384    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2" podUID="fe43caaf-eb11-4b75-a185-66ab33940e27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 19:08:38.610216 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:38.610103    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"]
Oct 30 19:08:38.611659 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:38.611582    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:08:38.611775 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:38.611711    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:08:38.611775 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:38.611761    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:08:39.043219 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:39.043176    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6" event={"ID":"0456a5cd-81b2-4dd1-9e36-a51808065ad0","Type":"ContainerStarted","Data":"2fd67f2c017f50ab596a11be49a591b307b0a278e1c0a7e63434ee36a8e8b255"}
Oct 30 19:08:39.043746 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:39.043230    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6" event={"ID":"0456a5cd-81b2-4dd1-9e36-a51808065ad0","Type":"ContainerStarted","Data":"2f0a733ee02c48ef0152db5a90e164d91a91e19557c8d2e9441f27484cadbc26"}
Oct 30 19:08:39.130940 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:39.130902    4544 patch_prober.go:29] interesting pod/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:08:39.131129 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:39.130965    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2" podUID="fe43caaf-eb11-4b75-a185-66ab33940e27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 30 19:08:39.138379 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:39.138353    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:39.140033 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:39.140008    4544 patch_prober.go:29] interesting pod/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:08:39.140148 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:39.140046    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6" podUID="0456a5cd-81b2-4dd1-9e36-a51808065ad0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 19:08:39.167420 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:39.167358    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6" podStartSLOduration=2.167340097 podStartE2EDuration="2.167340097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:39.161561963 +0000 UTC m=+1916.687437061" watchObservedRunningTime="2026-10-30 19:08:39.167340097 +0000 UTC m=+1916.693215194"
Oct 30 19:08:40.136108 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:40.136077    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:08:40.136517 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:40.136451    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:08:40.137265 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:40.137249    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:08:40.142662 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:40.142640    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:41.058845 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:41.058811    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:41.059676 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:41.059655    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:08:45.535234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.535197    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp"]
Oct 30 19:08:45.541011 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.540993    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp"
Oct 30 19:08:45.582276 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.582245    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp"]
Oct 30 19:08:45.595143 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.595114    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-071cf0\""
Oct 30 19:08:45.598114 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.598084    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmvk6\" (UniqueName: \"kubernetes.io/projected/04e84344-8275-4abf-b4de-61ff14cbd45c-kube-api-access-cmvk6\") pod \"payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp\" (UID: \"04e84344-8275-4abf-b4de-61ff14cbd45c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp"
Oct 30 19:08:45.598294 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.598274    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04e84344-8275-4abf-b4de-61ff14cbd45c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp\" (UID: \"04e84344-8275-4abf-b4de-61ff14cbd45c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp"
Oct 30 19:08:45.718393 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.718357    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d"]
Oct 30 19:08:45.742989 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.737313    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04e84344-8275-4abf-b4de-61ff14cbd45c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp\" (UID: \"04e84344-8275-4abf-b4de-61ff14cbd45c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp"
Oct 30 19:08:45.742989 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.737587    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmvk6\" (UniqueName: \"kubernetes.io/projected/04e84344-8275-4abf-b4de-61ff14cbd45c-kube-api-access-cmvk6\") pod \"payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp\" (UID: \"04e84344-8275-4abf-b4de-61ff14cbd45c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp"
Oct 30 19:08:45.742989 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.738798    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04e84344-8275-4abf-b4de-61ff14cbd45c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp\" (UID: \"04e84344-8275-4abf-b4de-61ff14cbd45c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp"
Oct 30 19:08:45.742989 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.740409    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d"
Oct 30 19:08:45.747100 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.747072    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d"]
Oct 30 19:08:45.807654 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.807564    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-071cf0-dockercfg-r2slh\""
Oct 30 19:08:45.818197 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.818143    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmvk6\" (UniqueName: \"kubernetes.io/projected/04e84344-8275-4abf-b4de-61ff14cbd45c-kube-api-access-cmvk6\") pod \"payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp\" (UID: \"04e84344-8275-4abf-b4de-61ff14cbd45c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp"
Oct 30 19:08:45.838855 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.838820    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw4rr\" (UniqueName: \"kubernetes.io/projected/57e25c31-3dbe-498f-b392-2d210ba51cd8-kube-api-access-fw4rr\") pod \"payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d\" (UID: \"57e25c31-3dbe-498f-b392-2d210ba51cd8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d"
Oct 30 19:08:45.839050 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.839032    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57e25c31-3dbe-498f-b392-2d210ba51cd8-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d\" (UID: \"57e25c31-3dbe-498f-b392-2d210ba51cd8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d"
Oct 30 19:08:45.854191 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.854162    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp"
Oct 30 19:08:45.943722 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.940349    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw4rr\" (UniqueName: \"kubernetes.io/projected/57e25c31-3dbe-498f-b392-2d210ba51cd8-kube-api-access-fw4rr\") pod \"payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d\" (UID: \"57e25c31-3dbe-498f-b392-2d210ba51cd8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d"
Oct 30 19:08:45.943722 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.940519    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57e25c31-3dbe-498f-b392-2d210ba51cd8-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d\" (UID: \"57e25c31-3dbe-498f-b392-2d210ba51cd8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d"
Oct 30 19:08:45.943722 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:45.941585    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57e25c31-3dbe-498f-b392-2d210ba51cd8-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d\" (UID: \"57e25c31-3dbe-498f-b392-2d210ba51cd8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d"
Oct 30 19:08:46.000622 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:46.000580    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw4rr\" (UniqueName: \"kubernetes.io/projected/57e25c31-3dbe-498f-b392-2d210ba51cd8-kube-api-access-fw4rr\") pod \"payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d\" (UID: \"57e25c31-3dbe-498f-b392-2d210ba51cd8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d"
Oct 30 19:08:46.069858 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:46.069787    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d"
Oct 30 19:08:46.252922 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:08:46.252893    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04e84344_8275_4abf_b4de_61ff14cbd45c.slice/crio-9e8b98c26c0aa8d4b6b13d2c9e432ffdeb7eb63e63fda948cc81d3ef2e55fb46 WatchSource:0}: Error finding container 9e8b98c26c0aa8d4b6b13d2c9e432ffdeb7eb63e63fda948cc81d3ef2e55fb46: Status 404 returned error can't find the container with id 9e8b98c26c0aa8d4b6b13d2c9e432ffdeb7eb63e63fda948cc81d3ef2e55fb46
Oct 30 19:08:46.255491 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:46.255460    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp"]
Oct 30 19:08:46.447803 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:08:46.447773    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57e25c31_3dbe_498f_b392_2d210ba51cd8.slice/crio-9e9b8eb12b6c1cfff38dcad0ea0574293532677b2c8ffc1dccf388876496d470 WatchSource:0}: Error finding container 9e9b8eb12b6c1cfff38dcad0ea0574293532677b2c8ffc1dccf388876496d470: Status 404 returned error can't find the container with id 9e9b8eb12b6c1cfff38dcad0ea0574293532677b2c8ffc1dccf388876496d470
Oct 30 19:08:46.450058 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:46.450032    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d"]
Oct 30 19:08:47.112242 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:47.112205    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d" event={"ID":"57e25c31-3dbe-498f-b392-2d210ba51cd8","Type":"ContainerStarted","Data":"88c0bfd9114be0f9be7c4edb08f8a0b410850cdc210bd135430e57ca62bfc78f"}
Oct 30 19:08:47.112764 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:47.112268    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d" event={"ID":"57e25c31-3dbe-498f-b392-2d210ba51cd8","Type":"ContainerStarted","Data":"9e9b8eb12b6c1cfff38dcad0ea0574293532677b2c8ffc1dccf388876496d470"}
Oct 30 19:08:47.112764 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:47.112301    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d"
Oct 30 19:08:47.113973 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:47.113942    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp" event={"ID":"04e84344-8275-4abf-b4de-61ff14cbd45c","Type":"ContainerStarted","Data":"f9594d6cd8356196fae5807cbdecde5151a57ba1390670de338b407f623a55fc"}
Oct 30 19:08:47.114099 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:47.113995    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp" event={"ID":"04e84344-8275-4abf-b4de-61ff14cbd45c","Type":"ContainerStarted","Data":"9e8b98c26c0aa8d4b6b13d2c9e432ffdeb7eb63e63fda948cc81d3ef2e55fb46"}
Oct 30 19:08:47.114099 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:47.114024    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp"
Oct 30 19:08:47.258970 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:47.258917    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d" podStartSLOduration=2.258902757 podStartE2EDuration="2.258902757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:47.253422518 +0000 UTC m=+1924.779297614" watchObservedRunningTime="2026-10-30 19:08:47.258902757 +0000 UTC m=+1924.784777853"
Oct 30 19:08:47.381555 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:47.377489    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp" podStartSLOduration=2.377471838 podStartE2EDuration="2.377471838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:47.373181292 +0000 UTC m=+1924.899056387" watchObservedRunningTime="2026-10-30 19:08:47.377471838 +0000 UTC m=+1924.903346935"
Oct 30 19:08:58.125056 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:58.124973    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp"
Oct 30 19:08:58.125495 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:08:58.125069    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d"
Oct 30 19:09:05.532878 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:09:05.532835    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:09:12.310398 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:12.308649    4544 patch_prober.go:29] interesting pod/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": read tcp 10.133.0.2:54196->10.133.0.103:15021: read: connection reset by peer" start-of-body=
Oct 30 19:09:12.310398 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:12.308751    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb" podUID="729b4f97-7d22-49ae-ac1b-4528af3b5242" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": read tcp 10.133.0.2:54196->10.133.0.103:15021: read: connection reset by peer"
Oct 30 19:09:12.367497 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:12.367471    4544 generic.go:356] "Generic (PLEG): container finished" podID="729b4f97-7d22-49ae-ac1b-4528af3b5242" containerID="571a3de3132365c3dc0b58d04e53ce7ad88e4c525ed948d1fabac6985a99a850" exitCode=0
Oct 30 19:09:12.367633 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:12.367554    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb" event={"ID":"729b4f97-7d22-49ae-ac1b-4528af3b5242","Type":"ContainerDied","Data":"571a3de3132365c3dc0b58d04e53ce7ad88e4c525ed948d1fabac6985a99a850"}
Oct 30 19:09:12.367934 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:12.367921    4544 scope.go:122] "RemoveContainer" containerID="571a3de3132365c3dc0b58d04e53ce7ad88e4c525ed948d1fabac6985a99a850"
Oct 30 19:09:12.369463 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:12.369432    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:09:12.369562 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:12.369522    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:09:12.369659 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:12.369569    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:09:13.400022 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:13.399979    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb" event={"ID":"729b4f97-7d22-49ae-ac1b-4528af3b5242","Type":"ContainerStarted","Data":"df2df1729f7deaa0eea3675f7c547382ac574c5107500bb4ea9c12b0f2003130"}
Oct 30 19:09:14.308111 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:14.308069    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:09:14.308317 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:14.308268    4544 patch_prober.go:29] interesting pod/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 19:09:14.308403 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:14.308329    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb" podUID="729b4f97-7d22-49ae-ac1b-4528af3b5242" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 19:09:15.308075 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:15.308037    4544 patch_prober.go:29] interesting pod/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 19:09:15.308479 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:15.308091    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb" podUID="729b4f97-7d22-49ae-ac1b-4528af3b5242" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 19:09:16.308326 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:16.308283    4544 patch_prober.go:29] interesting pod/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 19:09:16.308752 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:16.308350    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb" podUID="729b4f97-7d22-49ae-ac1b-4528af3b5242" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 19:09:17.307702 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:17.307665    4544 patch_prober.go:29] interesting pod/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 19:09:17.307881 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:17.307723    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb" podUID="729b4f97-7d22-49ae-ac1b-4528af3b5242" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 19:09:18.307592 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:18.307549    4544 patch_prober.go:29] interesting pod/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 30 19:09:18.308029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:18.307647    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb" podUID="729b4f97-7d22-49ae-ac1b-4528af3b5242" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 30 19:09:19.312814 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:19.312776    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:09:19.313653 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:19.313589    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:09:19.314369 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:19.314240    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:09:30.307276 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:30.307240    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"]
Oct 30 19:09:30.307784 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:30.307445    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc" podUID="4f494989-34da-4a63-95c2-7440b320bb0a" containerName="istio-proxy" containerID="cri-o://7e935715a3d50e14f720c7a53632b9a421a394e23296ca7de47bfc9b20944871" gracePeriod=30
Oct 30 19:09:34.602303 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:34.602263    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:09:34.602738 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:34.602332    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc" podUID="4f494989-34da-4a63-95c2-7440b320bb0a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:09:35.544518 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:09:35.544477    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:09:35.615423 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.615396    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:09:35.617367 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.617340    4544 generic.go:356] "Generic (PLEG): container finished" podID="4f494989-34da-4a63-95c2-7440b320bb0a" containerID="7e935715a3d50e14f720c7a53632b9a421a394e23296ca7de47bfc9b20944871" exitCode=0
Oct 30 19:09:35.617505 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.617382    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"
Oct 30 19:09:35.617505 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.617417    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc" event={"ID":"4f494989-34da-4a63-95c2-7440b320bb0a","Type":"ContainerDied","Data":"7e935715a3d50e14f720c7a53632b9a421a394e23296ca7de47bfc9b20944871"}
Oct 30 19:09:35.617505 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.617471    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc" event={"ID":"4f494989-34da-4a63-95c2-7440b320bb0a","Type":"ContainerDied","Data":"71bbde9d348a6f39fa08c3ebb325ede522be3fefb4f23998a59c03be88fe7418"}
Oct 30 19:09:35.617505 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.617497    4544 scope.go:122] "RemoveContainer" containerID="7e935715a3d50e14f720c7a53632b9a421a394e23296ca7de47bfc9b20944871"
Oct 30 19:09:35.627284 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.627271    4544 scope.go:122] "RemoveContainer" containerID="7e935715a3d50e14f720c7a53632b9a421a394e23296ca7de47bfc9b20944871"
Oct 30 19:09:35.627546 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:09:35.627525    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e935715a3d50e14f720c7a53632b9a421a394e23296ca7de47bfc9b20944871\": container with ID starting with 7e935715a3d50e14f720c7a53632b9a421a394e23296ca7de47bfc9b20944871 not found: ID does not exist" containerID="7e935715a3d50e14f720c7a53632b9a421a394e23296ca7de47bfc9b20944871"
Oct 30 19:09:35.627620 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.627560    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e935715a3d50e14f720c7a53632b9a421a394e23296ca7de47bfc9b20944871"} err="failed to get container status \"7e935715a3d50e14f720c7a53632b9a421a394e23296ca7de47bfc9b20944871\": rpc error: code = NotFound desc = could not find container \"7e935715a3d50e14f720c7a53632b9a421a394e23296ca7de47bfc9b20944871\": container with ID starting with 7e935715a3d50e14f720c7a53632b9a421a394e23296ca7de47bfc9b20944871 not found: ID does not exist"
Oct 30 19:09:35.672247 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.672223    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-istio-data\") pod \"4f494989-34da-4a63-95c2-7440b320bb0a\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") "
Oct 30 19:09:35.672380 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.672339    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f494989-34da-4a63-95c2-7440b320bb0a-kube-api-access-ndb58\" (UniqueName: \"kubernetes.io/projected/4f494989-34da-4a63-95c2-7440b320bb0a-kube-api-access-ndb58\") pod \"4f494989-34da-4a63-95c2-7440b320bb0a\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") "
Oct 30 19:09:35.672380 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.672359    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-workload-certs\") pod \"4f494989-34da-4a63-95c2-7440b320bb0a\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") "
Oct 30 19:09:35.672497 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.672413    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4f494989-34da-4a63-95c2-7440b320bb0a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f494989-34da-4a63-95c2-7440b320bb0a-istiod-ca-cert\") pod \"4f494989-34da-4a63-95c2-7440b320bb0a\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") "
Oct 30 19:09:35.672497 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.672440    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-credential-socket\") pod \"4f494989-34da-4a63-95c2-7440b320bb0a\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") "
Oct 30 19:09:35.672624 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.672487    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-istio-data" pod "4f494989-34da-4a63-95c2-7440b320bb0a" (UID: "4f494989-34da-4a63-95c2-7440b320bb0a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:35.672863 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.672817    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-workload-certs" pod "4f494989-34da-4a63-95c2-7440b320bb0a" (UID: "4f494989-34da-4a63-95c2-7440b320bb0a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:35.672974 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.672916    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-workload-socket\") pod \"4f494989-34da-4a63-95c2-7440b320bb0a\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") "
Oct 30 19:09:35.673037 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.673008    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-istio-envoy\") pod \"4f494989-34da-4a63-95c2-7440b320bb0a\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") "
Oct 30 19:09:35.673104 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.673088    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f494989-34da-4a63-95c2-7440b320bb0a-istio-token\" (UniqueName: \"kubernetes.io/projected/4f494989-34da-4a63-95c2-7440b320bb0a-istio-token\") pod \"4f494989-34da-4a63-95c2-7440b320bb0a\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") "
Oct 30 19:09:35.673218 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.673194    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-workload-socket" pod "4f494989-34da-4a63-95c2-7440b320bb0a" (UID: "4f494989-34da-4a63-95c2-7440b320bb0a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:35.673269 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.673207    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-credential-socket" pod "4f494989-34da-4a63-95c2-7440b320bb0a" (UID: "4f494989-34da-4a63-95c2-7440b320bb0a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:35.673269 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.673251    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4f494989-34da-4a63-95c2-7440b320bb0a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f494989-34da-4a63-95c2-7440b320bb0a-istio-podinfo\") pod \"4f494989-34da-4a63-95c2-7440b320bb0a\" (UID: \"4f494989-34da-4a63-95c2-7440b320bb0a\") "
Oct 30 19:09:35.674324 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.674279    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f494989-34da-4a63-95c2-7440b320bb0a-istiod-ca-cert" pod "4f494989-34da-4a63-95c2-7440b320bb0a" (UID: "4f494989-34da-4a63-95c2-7440b320bb0a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:09:35.676958 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.675787    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f494989-34da-4a63-95c2-7440b320bb0a-kube-api-access-ndb58" pod "4f494989-34da-4a63-95c2-7440b320bb0a" (UID: "4f494989-34da-4a63-95c2-7440b320bb0a"). InnerVolumeSpecName "kube-api-access-ndb58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:09:35.685781 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.685748    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:35.685781 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.685778    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f494989-34da-4a63-95c2-7440b320bb0a-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:35.685980 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.685797    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:35.685980 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.685813    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:35.685980 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.685831    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:35.694228 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.694192    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-istio-envoy" pod "4f494989-34da-4a63-95c2-7440b320bb0a" (UID: "4f494989-34da-4a63-95c2-7440b320bb0a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:35.694539 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.694517    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4f494989-34da-4a63-95c2-7440b320bb0a-istio-podinfo" pod "4f494989-34da-4a63-95c2-7440b320bb0a" (UID: "4f494989-34da-4a63-95c2-7440b320bb0a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:09:35.694669 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.694531    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f494989-34da-4a63-95c2-7440b320bb0a-istio-token" pod "4f494989-34da-4a63-95c2-7440b320bb0a" (UID: "4f494989-34da-4a63-95c2-7440b320bb0a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:09:35.789229 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.789195    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f494989-34da-4a63-95c2-7440b320bb0a-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:35.795266 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.795239    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndb58\" (UniqueName: \"kubernetes.io/projected/4f494989-34da-4a63-95c2-7440b320bb0a-kube-api-access-ndb58\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:35.795462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.795447    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f494989-34da-4a63-95c2-7440b320bb0a-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:35.795577 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:35.795544    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f494989-34da-4a63-95c2-7440b320bb0a-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:36.046830 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:36.046798    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"]
Oct 30 19:09:36.050857 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:36.050831    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-f85dc"]
Oct 30 19:09:36.607197 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:36.607164    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4f494989-34da-4a63-95c2-7440b320bb0a" path="/var/lib/kubelet/pods/4f494989-34da-4a63-95c2-7440b320bb0a/volumes"
Oct 30 19:09:49.677246 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:49.677208    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv"]
Oct 30 19:09:49.786255 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:49.786219    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs"]
Oct 30 19:09:54.677941 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:54.677879    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv" podUID="9877e03e-92b6-43c7-83c7-6fdea1d04000" containerName="payload-processing" containerID="cri-o://3dbbe599a4cbde55c2c31d4a0c37792a5fb4dba3670982e9bb7c64cbc050c138" gracePeriod=25
Oct 30 19:09:54.787057 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:54.787007    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs" podUID="8955fdbb-df1e-4a72-83c2-bef3dbb86a68" containerName="payload-pre-processing" containerID="cri-o://5a6fac3f1d0c3a0b477d694daeaa82ce2a1d94302adedaf07ac695a0d96d1016" gracePeriod=25
Oct 30 19:09:59.404033 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:09:59.403991    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs" podUID="8955fdbb-df1e-4a72-83c2-bef3dbb86a68" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:10:00.420710 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:00.420673    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv" podUID="9877e03e-92b6-43c7-83c7-6fdea1d04000" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:10:05.596629 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:10:05.586894    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:10:09.406747 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:09.406579    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs" podUID="8955fdbb-df1e-4a72-83c2-bef3dbb86a68" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:10:10.422737 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:10.422694    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv" podUID="9877e03e-92b6-43c7-83c7-6fdea1d04000" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:10:11.423811 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:11.423736    4544 patch_prober.go:29] interesting pod/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:10:11.424295 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:11.423843    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr" podUID="b1badcf1-f26a-4a5e-a068-678c9971e21c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:10:13.026888 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:13.026855    4544 generic.go:356] "Generic (PLEG): container finished" podID="b1badcf1-f26a-4a5e-a068-678c9971e21c" containerID="33e8b84fc9d65eb9bbcb59ec7f1ead7c6a97f19e1594a6eb8517062a87c74171" exitCode=0
Oct 30 19:10:13.027340 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:13.026926    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr" event={"ID":"b1badcf1-f26a-4a5e-a068-678c9971e21c","Type":"ContainerDied","Data":"33e8b84fc9d65eb9bbcb59ec7f1ead7c6a97f19e1594a6eb8517062a87c74171"}
Oct 30 19:10:13.027409 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:13.027374    4544 scope.go:122] "RemoveContainer" containerID="33e8b84fc9d65eb9bbcb59ec7f1ead7c6a97f19e1594a6eb8517062a87c74171"
Oct 30 19:10:13.028788 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:13.028765    4544 generic.go:356] "Generic (PLEG): container finished" podID="a70cfd95-bd84-4720-ae1a-630761e61fb1" containerID="5858631fe9c06128edf98eb170e84e080c068edc31951644d87a67d6f64fe3e8" exitCode=0
Oct 30 19:10:13.028908 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:13.028835    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm" event={"ID":"a70cfd95-bd84-4720-ae1a-630761e61fb1","Type":"ContainerDied","Data":"5858631fe9c06128edf98eb170e84e080c068edc31951644d87a67d6f64fe3e8"}
Oct 30 19:10:13.029204 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:13.029191    4544 scope.go:122] "RemoveContainer" containerID="5858631fe9c06128edf98eb170e84e080c068edc31951644d87a67d6f64fe3e8"
Oct 30 19:10:13.029362 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:13.029337    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:10:13.029416 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:13.029404    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:10:13.029451 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:13.029438    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:10:13.030901 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:13.030874    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:10:13.030978 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:13.030951    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:10:13.031016 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:13.030997    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:10:14.243624 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:14.243340    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr" event={"ID":"b1badcf1-f26a-4a5e-a068-678c9971e21c","Type":"ContainerStarted","Data":"4a12a3cc0f0de8c7798f0a3039d44161999c009dba82d1c1881e9a43711833af"}
Oct 30 19:10:14.335158 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:14.335120    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm" event={"ID":"a70cfd95-bd84-4720-ae1a-630761e61fb1","Type":"ContainerStarted","Data":"1610b2fc724d58f45c4284141b554183a2face02aa6f6f895da68b2328b87de3"}
Oct 30 19:10:14.542625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:14.541634    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:10:14.567655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:14.550929    4544 patch_prober.go:29] interesting pod/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 19:10:14.567655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:14.550997    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr" podUID="b1badcf1-f26a-4a5e-a068-678c9971e21c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 19:10:14.881627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:14.879814    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:10:14.888630 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:14.884749    4544 patch_prober.go:29] interesting pod/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:10:14.888630 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:14.884800    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm" podUID="a70cfd95-bd84-4720-ae1a-630761e61fb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 19:10:15.229744 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.229705    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv"
Oct 30 19:10:15.323303 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.316510    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9877e03e-92b6-43c7-83c7-6fdea1d04000-kube-api-access-sw2bw\" (UniqueName: \"kubernetes.io/projected/9877e03e-92b6-43c7-83c7-6fdea1d04000-kube-api-access-sw2bw\") pod \"9877e03e-92b6-43c7-83c7-6fdea1d04000\" (UID: \"9877e03e-92b6-43c7-83c7-6fdea1d04000\") "
Oct 30 19:10:15.323303 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.316580    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9877e03e-92b6-43c7-83c7-6fdea1d04000-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9877e03e-92b6-43c7-83c7-6fdea1d04000-plugins-config-volume\") pod \"9877e03e-92b6-43c7-83c7-6fdea1d04000\" (UID: \"9877e03e-92b6-43c7-83c7-6fdea1d04000\") "
Oct 30 19:10:15.323303 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.318004    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9877e03e-92b6-43c7-83c7-6fdea1d04000-plugins-config-volume" pod "9877e03e-92b6-43c7-83c7-6fdea1d04000" (UID: "9877e03e-92b6-43c7-83c7-6fdea1d04000"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:10:15.323303 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.322027    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9877e03e-92b6-43c7-83c7-6fdea1d04000-kube-api-access-sw2bw" pod "9877e03e-92b6-43c7-83c7-6fdea1d04000" (UID: "9877e03e-92b6-43c7-83c7-6fdea1d04000"). InnerVolumeSpecName "kube-api-access-sw2bw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:10:15.336134 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.336080    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs"
Oct 30 19:10:15.344737 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.344709    4544 generic.go:356] "Generic (PLEG): container finished" podID="8955fdbb-df1e-4a72-83c2-bef3dbb86a68" containerID="5a6fac3f1d0c3a0b477d694daeaa82ce2a1d94302adedaf07ac695a0d96d1016" exitCode=0
Oct 30 19:10:15.344886 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.344754    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs"
Oct 30 19:10:15.344886 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.344788    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs" event={"ID":"8955fdbb-df1e-4a72-83c2-bef3dbb86a68","Type":"ContainerDied","Data":"5a6fac3f1d0c3a0b477d694daeaa82ce2a1d94302adedaf07ac695a0d96d1016"}
Oct 30 19:10:15.344886 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.344836    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs" event={"ID":"8955fdbb-df1e-4a72-83c2-bef3dbb86a68","Type":"ContainerDied","Data":"b5f991ee7349faee14d35991c7e6b84feb6c118e2cb3ab0ebc39531e1bc60b4d"}
Oct 30 19:10:15.344886 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.344866    4544 scope.go:122] "RemoveContainer" containerID="5a6fac3f1d0c3a0b477d694daeaa82ce2a1d94302adedaf07ac695a0d96d1016"
Oct 30 19:10:15.346830 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.346810    4544 generic.go:356] "Generic (PLEG): container finished" podID="9877e03e-92b6-43c7-83c7-6fdea1d04000" containerID="3dbbe599a4cbde55c2c31d4a0c37792a5fb4dba3670982e9bb7c64cbc050c138" exitCode=0
Oct 30 19:10:15.347861 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.347845    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv"
Oct 30 19:10:15.349830 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.349806    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv" event={"ID":"9877e03e-92b6-43c7-83c7-6fdea1d04000","Type":"ContainerDied","Data":"3dbbe599a4cbde55c2c31d4a0c37792a5fb4dba3670982e9bb7c64cbc050c138"}
Oct 30 19:10:15.349920 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.349862    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv" event={"ID":"9877e03e-92b6-43c7-83c7-6fdea1d04000","Type":"ContainerDied","Data":"650e368f04292b256a21be8a05ef5aaee9dd93078a92a3950e52c80e187a84f9"}
Oct 30 19:10:15.356433 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.356414    4544 scope.go:122] "RemoveContainer" containerID="5a6fac3f1d0c3a0b477d694daeaa82ce2a1d94302adedaf07ac695a0d96d1016"
Oct 30 19:10:15.361622 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:10:15.356751    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a6fac3f1d0c3a0b477d694daeaa82ce2a1d94302adedaf07ac695a0d96d1016\": container with ID starting with 5a6fac3f1d0c3a0b477d694daeaa82ce2a1d94302adedaf07ac695a0d96d1016 not found: ID does not exist" containerID="5a6fac3f1d0c3a0b477d694daeaa82ce2a1d94302adedaf07ac695a0d96d1016"
Oct 30 19:10:15.361622 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.356790    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a6fac3f1d0c3a0b477d694daeaa82ce2a1d94302adedaf07ac695a0d96d1016"} err="failed to get container status \"5a6fac3f1d0c3a0b477d694daeaa82ce2a1d94302adedaf07ac695a0d96d1016\": rpc error: code = NotFound desc = could not find container \"5a6fac3f1d0c3a0b477d694daeaa82ce2a1d94302adedaf07ac695a0d96d1016\": container with ID starting with 5a6fac3f1d0c3a0b477d694daeaa82ce2a1d94302adedaf07ac695a0d96d1016 not found: ID does not exist"
Oct 30 19:10:15.361622 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.356813    4544 scope.go:122] "RemoveContainer" containerID="3dbbe599a4cbde55c2c31d4a0c37792a5fb4dba3670982e9bb7c64cbc050c138"
Oct 30 19:10:15.434512 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.425172    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8955fdbb-df1e-4a72-83c2-bef3dbb86a68-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8955fdbb-df1e-4a72-83c2-bef3dbb86a68-plugins-config-volume\") pod \"8955fdbb-df1e-4a72-83c2-bef3dbb86a68\" (UID: \"8955fdbb-df1e-4a72-83c2-bef3dbb86a68\") "
Oct 30 19:10:15.434512 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.425320    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8955fdbb-df1e-4a72-83c2-bef3dbb86a68-kube-api-access-n7bmd\" (UniqueName: \"kubernetes.io/projected/8955fdbb-df1e-4a72-83c2-bef3dbb86a68-kube-api-access-n7bmd\") pod \"8955fdbb-df1e-4a72-83c2-bef3dbb86a68\" (UID: \"8955fdbb-df1e-4a72-83c2-bef3dbb86a68\") "
Oct 30 19:10:15.434512 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.426238    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sw2bw\" (UniqueName: \"kubernetes.io/projected/9877e03e-92b6-43c7-83c7-6fdea1d04000-kube-api-access-sw2bw\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:15.434512 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.426265    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9877e03e-92b6-43c7-83c7-6fdea1d04000-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:15.466647 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.465194    4544 patch_prober.go:29] interesting pod/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 19:10:15.466647 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.465251    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr" podUID="b1badcf1-f26a-4a5e-a068-678c9971e21c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 19:10:15.466647 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.466264    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8955fdbb-df1e-4a72-83c2-bef3dbb86a68-plugins-config-volume" pod "8955fdbb-df1e-4a72-83c2-bef3dbb86a68" (UID: "8955fdbb-df1e-4a72-83c2-bef3dbb86a68"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:10:15.493626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.477295    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8955fdbb-df1e-4a72-83c2-bef3dbb86a68-kube-api-access-n7bmd" pod "8955fdbb-df1e-4a72-83c2-bef3dbb86a68" (UID: "8955fdbb-df1e-4a72-83c2-bef3dbb86a68"). InnerVolumeSpecName "kube-api-access-n7bmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:10:15.532633 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.531908    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8955fdbb-df1e-4a72-83c2-bef3dbb86a68-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:15.532633 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.531947    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7bmd\" (UniqueName: \"kubernetes.io/projected/8955fdbb-df1e-4a72-83c2-bef3dbb86a68-kube-api-access-n7bmd\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:15.541803 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.541245    4544 scope.go:122] "RemoveContainer" containerID="3dbbe599a4cbde55c2c31d4a0c37792a5fb4dba3670982e9bb7c64cbc050c138"
Oct 30 19:10:15.541803 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:10:15.541674    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dbbe599a4cbde55c2c31d4a0c37792a5fb4dba3670982e9bb7c64cbc050c138\": container with ID starting with 3dbbe599a4cbde55c2c31d4a0c37792a5fb4dba3670982e9bb7c64cbc050c138 not found: ID does not exist" containerID="3dbbe599a4cbde55c2c31d4a0c37792a5fb4dba3670982e9bb7c64cbc050c138"
Oct 30 19:10:15.541803 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.541707    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dbbe599a4cbde55c2c31d4a0c37792a5fb4dba3670982e9bb7c64cbc050c138"} err="failed to get container status \"3dbbe599a4cbde55c2c31d4a0c37792a5fb4dba3670982e9bb7c64cbc050c138\": rpc error: code = NotFound desc = could not find container \"3dbbe599a4cbde55c2c31d4a0c37792a5fb4dba3670982e9bb7c64cbc050c138\": container with ID starting with 3dbbe599a4cbde55c2c31d4a0c37792a5fb4dba3670982e9bb7c64cbc050c138 not found: ID does not exist"
Oct 30 19:10:15.744067 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.744035    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv"]
Oct 30 19:10:15.747543 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.747515    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-314bd6fa-77d6858c46-rqwjv"]
Oct 30 19:10:15.879627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.879401    4544 patch_prober.go:29] interesting pod/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:10:15.879627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.879457    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm" podUID="a70cfd95-bd84-4720-ae1a-630761e61fb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 19:10:15.879888 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.879747    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs"]
Oct 30 19:10:15.934675 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:15.933523    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-314bd6fa-7c6745c447-kwmvs"]
Oct 30 19:10:16.455624 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:16.454360    4544 patch_prober.go:29] interesting pod/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 19:10:16.455624 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:16.454406    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr" podUID="b1badcf1-f26a-4a5e-a068-678c9971e21c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 19:10:16.616652 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:16.615747    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8955fdbb-df1e-4a72-83c2-bef3dbb86a68" path="/var/lib/kubelet/pods/8955fdbb-df1e-4a72-83c2-bef3dbb86a68/volumes"
Oct 30 19:10:16.616652 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:16.616225    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9877e03e-92b6-43c7-83c7-6fdea1d04000" path="/var/lib/kubelet/pods/9877e03e-92b6-43c7-83c7-6fdea1d04000/volumes"
Oct 30 19:10:16.877624 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:16.876969    4544 patch_prober.go:29] interesting pod/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:10:16.877624 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:16.877034    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm" podUID="a70cfd95-bd84-4720-ae1a-630761e61fb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 19:10:17.455167 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:17.455124    4544 patch_prober.go:29] interesting pod/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 19:10:17.455365 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:17.455171    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr" podUID="b1badcf1-f26a-4a5e-a068-678c9971e21c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 19:10:17.883004 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:17.882968    4544 patch_prober.go:29] interesting pod/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:10:17.884014 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:17.883589    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm" podUID="a70cfd95-bd84-4720-ae1a-630761e61fb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 19:10:18.455199 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:18.455167    4544 patch_prober.go:29] interesting pod/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 19:10:18.455379 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:18.455216    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr" podUID="b1badcf1-f26a-4a5e-a068-678c9971e21c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 19:10:18.880698 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:18.877315    4544 patch_prober.go:29] interesting pod/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:10:18.880698 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:18.877376    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm" podUID="a70cfd95-bd84-4720-ae1a-630761e61fb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 19:10:19.455175 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:19.455144    4544 patch_prober.go:29] interesting pod/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 19:10:19.455584 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:19.455193    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr" podUID="b1badcf1-f26a-4a5e-a068-678c9971e21c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 19:10:19.879661 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:19.879630    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:10:19.880102 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:19.880081    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:10:19.880907 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:19.880884    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:10:20.460966 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:20.460934    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:10:20.461493 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:20.461470    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:10:20.462117 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:20.462096    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:10:35.532064 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:10:35.532029    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:10:39.625756 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:39.625726    4544 generic.go:356] "Generic (PLEG): container finished" podID="0456a5cd-81b2-4dd1-9e36-a51808065ad0" containerID="2fd67f2c017f50ab596a11be49a591b307b0a278e1c0a7e63434ee36a8e8b255" exitCode=0
Oct 30 19:10:39.626240 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:39.625784    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6" event={"ID":"0456a5cd-81b2-4dd1-9e36-a51808065ad0","Type":"ContainerDied","Data":"2fd67f2c017f50ab596a11be49a591b307b0a278e1c0a7e63434ee36a8e8b255"}
Oct 30 19:10:39.626240 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:39.626096    4544 scope.go:122] "RemoveContainer" containerID="2fd67f2c017f50ab596a11be49a591b307b0a278e1c0a7e63434ee36a8e8b255"
Oct 30 19:10:39.627659 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:39.627596    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:10:39.627776 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:39.627698    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:10:39.627776 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:39.627731    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:10:40.634234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:40.634201    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6" event={"ID":"0456a5cd-81b2-4dd1-9e36-a51808065ad0","Type":"ContainerStarted","Data":"5ec6d6e945b63f3120169945aa0a7bf106daea4bb0e5ee335c887551f0d04265"}
Oct 30 19:10:41.059631 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:41.058692    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:10:41.139054 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:41.139010    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:10:41.141165 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:41.141129    4544 patch_prober.go:29] interesting pod/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:10:41.141303 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:41.141224    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6" podUID="0456a5cd-81b2-4dd1-9e36-a51808065ad0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 19:10:42.139271 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:42.139234    4544 patch_prober.go:29] interesting pod/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:10:42.139767 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:42.139283    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6" podUID="0456a5cd-81b2-4dd1-9e36-a51808065ad0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 19:10:43.139215 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:43.139179    4544 patch_prober.go:29] interesting pod/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:10:43.139360 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:43.139242    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6" podUID="0456a5cd-81b2-4dd1-9e36-a51808065ad0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 19:10:44.138616 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:44.138568    4544 patch_prober.go:29] interesting pod/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:10:44.138803 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:44.138647    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6" podUID="0456a5cd-81b2-4dd1-9e36-a51808065ad0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 19:10:45.139011 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:45.138978    4544 patch_prober.go:29] interesting pod/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:10:45.139431 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:45.139040    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6" podUID="0456a5cd-81b2-4dd1-9e36-a51808065ad0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 19:10:46.143546 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:46.143514    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:10:46.144778 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:10:46.144755    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:11:02.718038 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:02.718004    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48"]
Oct 30 19:11:02.824132 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:02.824100    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk"]
Oct 30 19:11:05.531896 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:11:05.531857    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:11:07.385472 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:07.385424    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb"]
Oct 30 19:11:07.719023 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:07.718943    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48" podUID="50b8d8f8-30ae-4c27-9e2f-44bd646c916d" containerName="payload-processing" containerID="cri-o://7d6257f568a7532dda61d4b5993b9b74ac47c7f8dfe55bbfdbf3930e0bc61ec0" gracePeriod=25
Oct 30 19:11:07.824562 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:07.824500    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk" podUID="76db530e-e95b-498c-be91-e4bb9291fa43" containerName="payload-pre-processing" containerID="cri-o://5165f948361c23c51dd3bb85f0d14d05b1acf695e17080dbb3a4ad984bcabd1a" gracePeriod=25
Oct 30 19:11:08.038388 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.037956    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48"
Oct 30 19:11:08.122635 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.122443    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50b8d8f8-30ae-4c27-9e2f-44bd646c916d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50b8d8f8-30ae-4c27-9e2f-44bd646c916d-plugins-config-volume\") pod \"50b8d8f8-30ae-4c27-9e2f-44bd646c916d\" (UID: \"50b8d8f8-30ae-4c27-9e2f-44bd646c916d\") "
Oct 30 19:11:08.122635 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.122493    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50b8d8f8-30ae-4c27-9e2f-44bd646c916d-kube-api-access-55fmn\" (UniqueName: \"kubernetes.io/projected/50b8d8f8-30ae-4c27-9e2f-44bd646c916d-kube-api-access-55fmn\") pod \"50b8d8f8-30ae-4c27-9e2f-44bd646c916d\" (UID: \"50b8d8f8-30ae-4c27-9e2f-44bd646c916d\") "
Oct 30 19:11:08.122836 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.122794    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50b8d8f8-30ae-4c27-9e2f-44bd646c916d-plugins-config-volume" pod "50b8d8f8-30ae-4c27-9e2f-44bd646c916d" (UID: "50b8d8f8-30ae-4c27-9e2f-44bd646c916d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:11:08.123250 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.123224    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50b8d8f8-30ae-4c27-9e2f-44bd646c916d-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:11:08.132422 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.132396    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50b8d8f8-30ae-4c27-9e2f-44bd646c916d-kube-api-access-55fmn" pod "50b8d8f8-30ae-4c27-9e2f-44bd646c916d" (UID: "50b8d8f8-30ae-4c27-9e2f-44bd646c916d"). InnerVolumeSpecName "kube-api-access-55fmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:11:08.143598 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.143579    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk"
Oct 30 19:11:08.224545 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.224516    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76db530e-e95b-498c-be91-e4bb9291fa43-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76db530e-e95b-498c-be91-e4bb9291fa43-plugins-config-volume\") pod \"76db530e-e95b-498c-be91-e4bb9291fa43\" (UID: \"76db530e-e95b-498c-be91-e4bb9291fa43\") "
Oct 30 19:11:08.224719 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.224708    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76db530e-e95b-498c-be91-e4bb9291fa43-kube-api-access-6frwj\" (UniqueName: \"kubernetes.io/projected/76db530e-e95b-498c-be91-e4bb9291fa43-kube-api-access-6frwj\") pod \"76db530e-e95b-498c-be91-e4bb9291fa43\" (UID: \"76db530e-e95b-498c-be91-e4bb9291fa43\") "
Oct 30 19:11:08.224902 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.224881    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76db530e-e95b-498c-be91-e4bb9291fa43-plugins-config-volume" pod "76db530e-e95b-498c-be91-e4bb9291fa43" (UID: "76db530e-e95b-498c-be91-e4bb9291fa43"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:11:08.225193 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.225177    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55fmn\" (UniqueName: \"kubernetes.io/projected/50b8d8f8-30ae-4c27-9e2f-44bd646c916d-kube-api-access-55fmn\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:11:08.225239 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.225196    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76db530e-e95b-498c-be91-e4bb9291fa43-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:11:08.226740 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.226719    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76db530e-e95b-498c-be91-e4bb9291fa43-kube-api-access-6frwj" pod "76db530e-e95b-498c-be91-e4bb9291fa43" (UID: "76db530e-e95b-498c-be91-e4bb9291fa43"). InnerVolumeSpecName "kube-api-access-6frwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:11:08.326612 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.326562    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6frwj\" (UniqueName: \"kubernetes.io/projected/76db530e-e95b-498c-be91-e4bb9291fa43-kube-api-access-6frwj\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:11:08.897255 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.897145    4544 generic.go:356] "Generic (PLEG): container finished" podID="50b8d8f8-30ae-4c27-9e2f-44bd646c916d" containerID="7d6257f568a7532dda61d4b5993b9b74ac47c7f8dfe55bbfdbf3930e0bc61ec0" exitCode=0
Oct 30 19:11:08.897948 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.897922    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48" event={"ID":"50b8d8f8-30ae-4c27-9e2f-44bd646c916d","Type":"ContainerDied","Data":"7d6257f568a7532dda61d4b5993b9b74ac47c7f8dfe55bbfdbf3930e0bc61ec0"}
Oct 30 19:11:08.898156 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.898132    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48" event={"ID":"50b8d8f8-30ae-4c27-9e2f-44bd646c916d","Type":"ContainerDied","Data":"1a9bb383777901b29265b087b1fe6a846108e93f54d406ceed458252703da151"}
Oct 30 19:11:08.898290 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.898277    4544 scope.go:122] "RemoveContainer" containerID="7d6257f568a7532dda61d4b5993b9b74ac47c7f8dfe55bbfdbf3930e0bc61ec0"
Oct 30 19:11:08.898586 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.898571    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48"
Oct 30 19:11:08.914359 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.914318    4544 generic.go:356] "Generic (PLEG): container finished" podID="76db530e-e95b-498c-be91-e4bb9291fa43" containerID="5165f948361c23c51dd3bb85f0d14d05b1acf695e17080dbb3a4ad984bcabd1a" exitCode=0
Oct 30 19:11:08.914671 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.914648    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk" event={"ID":"76db530e-e95b-498c-be91-e4bb9291fa43","Type":"ContainerDied","Data":"5165f948361c23c51dd3bb85f0d14d05b1acf695e17080dbb3a4ad984bcabd1a"}
Oct 30 19:11:08.914879 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.914845    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk" event={"ID":"76db530e-e95b-498c-be91-e4bb9291fa43","Type":"ContainerDied","Data":"a62871c33d75bcf65ca1b9ba9cce8591966ba508e68e63e2b5819bfdd326240e"}
Oct 30 19:11:08.915109 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.915093    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk"
Oct 30 19:11:08.937957 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.937877    4544 scope.go:122] "RemoveContainer" containerID="7d6257f568a7532dda61d4b5993b9b74ac47c7f8dfe55bbfdbf3930e0bc61ec0"
Oct 30 19:11:08.941387 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:11:08.941332    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d6257f568a7532dda61d4b5993b9b74ac47c7f8dfe55bbfdbf3930e0bc61ec0\": container with ID starting with 7d6257f568a7532dda61d4b5993b9b74ac47c7f8dfe55bbfdbf3930e0bc61ec0 not found: ID does not exist" containerID="7d6257f568a7532dda61d4b5993b9b74ac47c7f8dfe55bbfdbf3930e0bc61ec0"
Oct 30 19:11:08.941495 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.941400    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d6257f568a7532dda61d4b5993b9b74ac47c7f8dfe55bbfdbf3930e0bc61ec0"} err="failed to get container status \"7d6257f568a7532dda61d4b5993b9b74ac47c7f8dfe55bbfdbf3930e0bc61ec0\": rpc error: code = NotFound desc = could not find container \"7d6257f568a7532dda61d4b5993b9b74ac47c7f8dfe55bbfdbf3930e0bc61ec0\": container with ID starting with 7d6257f568a7532dda61d4b5993b9b74ac47c7f8dfe55bbfdbf3930e0bc61ec0 not found: ID does not exist"
Oct 30 19:11:08.941495 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:08.941436    4544 scope.go:122] "RemoveContainer" containerID="5165f948361c23c51dd3bb85f0d14d05b1acf695e17080dbb3a4ad984bcabd1a"
Oct 30 19:11:09.009660 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:09.004975    4544 scope.go:122] "RemoveContainer" containerID="5165f948361c23c51dd3bb85f0d14d05b1acf695e17080dbb3a4ad984bcabd1a"
Oct 30 19:11:09.009660 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:11:09.005572    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5165f948361c23c51dd3bb85f0d14d05b1acf695e17080dbb3a4ad984bcabd1a\": container with ID starting with 5165f948361c23c51dd3bb85f0d14d05b1acf695e17080dbb3a4ad984bcabd1a not found: ID does not exist" containerID="5165f948361c23c51dd3bb85f0d14d05b1acf695e17080dbb3a4ad984bcabd1a"
Oct 30 19:11:09.009660 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:09.005623    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5165f948361c23c51dd3bb85f0d14d05b1acf695e17080dbb3a4ad984bcabd1a"} err="failed to get container status \"5165f948361c23c51dd3bb85f0d14d05b1acf695e17080dbb3a4ad984bcabd1a\": rpc error: code = NotFound desc = could not find container \"5165f948361c23c51dd3bb85f0d14d05b1acf695e17080dbb3a4ad984bcabd1a\": container with ID starting with 5165f948361c23c51dd3bb85f0d14d05b1acf695e17080dbb3a4ad984bcabd1a not found: ID does not exist"
Oct 30 19:11:09.083889 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:09.083857    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk"]
Oct 30 19:11:09.126854 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:09.126818    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cfb433-54998b7d6c-mj2lk"]
Oct 30 19:11:09.193669 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:09.193587    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48"]
Oct 30 19:11:09.198987 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:09.198961    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cfb433-75c4d965b4-88c48"]
Oct 30 19:11:10.605793 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:10.605749    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="50b8d8f8-30ae-4c27-9e2f-44bd646c916d" path="/var/lib/kubelet/pods/50b8d8f8-30ae-4c27-9e2f-44bd646c916d/volumes"
Oct 30 19:11:10.606368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:10.606347    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="76db530e-e95b-498c-be91-e4bb9291fa43" path="/var/lib/kubelet/pods/76db530e-e95b-498c-be91-e4bb9291fa43/volumes"
Oct 30 19:11:12.386427 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:12.386226    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb" podUID="1f93fb78-4ca3-4012-8d92-9f98d8747614" containerName="payload-pre-processing" containerID="cri-o://105210997c89cc650c77505ab273d32969a136801d2989d9bcfb10de2946b835" gracePeriod=25
Oct 30 19:11:12.893326 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:12.892882    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb"
Oct 30 19:11:12.954422 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:12.954388    4544 generic.go:356] "Generic (PLEG): container finished" podID="1f93fb78-4ca3-4012-8d92-9f98d8747614" containerID="105210997c89cc650c77505ab273d32969a136801d2989d9bcfb10de2946b835" exitCode=0
Oct 30 19:11:12.954422 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:12.954426    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb"
Oct 30 19:11:12.954683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:12.954463    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb" event={"ID":"1f93fb78-4ca3-4012-8d92-9f98d8747614","Type":"ContainerDied","Data":"105210997c89cc650c77505ab273d32969a136801d2989d9bcfb10de2946b835"}
Oct 30 19:11:12.954683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:12.954513    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb" event={"ID":"1f93fb78-4ca3-4012-8d92-9f98d8747614","Type":"ContainerDied","Data":"c46a09be5bcf54718bb0373e25e39f0ba03217beb536d6bc9ca19ac9ddb2d1db"}
Oct 30 19:11:12.954683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:12.954537    4544 scope.go:122] "RemoveContainer" containerID="105210997c89cc650c77505ab273d32969a136801d2989d9bcfb10de2946b835"
Oct 30 19:11:12.965161 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:12.965145    4544 scope.go:122] "RemoveContainer" containerID="105210997c89cc650c77505ab273d32969a136801d2989d9bcfb10de2946b835"
Oct 30 19:11:12.965427 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:11:12.965407    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"105210997c89cc650c77505ab273d32969a136801d2989d9bcfb10de2946b835\": container with ID starting with 105210997c89cc650c77505ab273d32969a136801d2989d9bcfb10de2946b835 not found: ID does not exist" containerID="105210997c89cc650c77505ab273d32969a136801d2989d9bcfb10de2946b835"
Oct 30 19:11:12.965495 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:12.965434    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"105210997c89cc650c77505ab273d32969a136801d2989d9bcfb10de2946b835"} err="failed to get container status \"105210997c89cc650c77505ab273d32969a136801d2989d9bcfb10de2946b835\": rpc error: code = NotFound desc = could not find container \"105210997c89cc650c77505ab273d32969a136801d2989d9bcfb10de2946b835\": container with ID starting with 105210997c89cc650c77505ab273d32969a136801d2989d9bcfb10de2946b835 not found: ID does not exist"
Oct 30 19:11:13.036059 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:13.035980    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f93fb78-4ca3-4012-8d92-9f98d8747614-kube-api-access-vtxmh\" (UniqueName: \"kubernetes.io/projected/1f93fb78-4ca3-4012-8d92-9f98d8747614-kube-api-access-vtxmh\") pod \"1f93fb78-4ca3-4012-8d92-9f98d8747614\" (UID: \"1f93fb78-4ca3-4012-8d92-9f98d8747614\") "
Oct 30 19:11:13.036287 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:13.036272    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f93fb78-4ca3-4012-8d92-9f98d8747614-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f93fb78-4ca3-4012-8d92-9f98d8747614-plugins-config-volume\") pod \"1f93fb78-4ca3-4012-8d92-9f98d8747614\" (UID: \"1f93fb78-4ca3-4012-8d92-9f98d8747614\") "
Oct 30 19:11:13.036709 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:13.036688    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f93fb78-4ca3-4012-8d92-9f98d8747614-plugins-config-volume" pod "1f93fb78-4ca3-4012-8d92-9f98d8747614" (UID: "1f93fb78-4ca3-4012-8d92-9f98d8747614"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:11:13.036816 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:13.036802    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f93fb78-4ca3-4012-8d92-9f98d8747614-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:11:13.038037 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:13.038016    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f93fb78-4ca3-4012-8d92-9f98d8747614-kube-api-access-vtxmh" pod "1f93fb78-4ca3-4012-8d92-9f98d8747614" (UID: "1f93fb78-4ca3-4012-8d92-9f98d8747614"). InnerVolumeSpecName "kube-api-access-vtxmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:11:13.138023 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:13.137988    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtxmh\" (UniqueName: \"kubernetes.io/projected/1f93fb78-4ca3-4012-8d92-9f98d8747614-kube-api-access-vtxmh\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:11:13.379914 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:13.379835    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb"]
Oct 30 19:11:13.383029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:13.383001    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-cc079d-5955cccdf9-lgvdb"]
Oct 30 19:11:14.604100 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:14.604064    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f93fb78-4ca3-4012-8d92-9f98d8747614" path="/var/lib/kubelet/pods/1f93fb78-4ca3-4012-8d92-9f98d8747614/volumes"
Oct 30 19:11:35.534010 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:11:35.533976    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:11:42.896904 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:42.896877    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 19:11:42.898868 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:42.898846    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 19:11:43.214566 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:43.214489    4544 scope.go:122] "RemoveContainer" containerID="f4647b78b4c3cf9b4e0b25c05c56f1b48c03e9bcd979b21c71989d34c62c17a6"
Oct 30 19:11:53.398389 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:53.398310    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d"]
Oct 30 19:11:53.506681 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:53.506643    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp"]
Oct 30 19:11:58.399265 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:58.399211    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d" podUID="57e25c31-3dbe-498f-b392-2d210ba51cd8" containerName="payload-processing" containerID="cri-o://88c0bfd9114be0f9be7c4edb08f8a0b410850cdc210bd135430e57ca62bfc78f" gracePeriod=25
Oct 30 19:11:58.507938 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:58.507899    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp" podUID="04e84344-8275-4abf-b4de-61ff14cbd45c" containerName="payload-pre-processing" containerID="cri-o://f9594d6cd8356196fae5807cbdecde5151a57ba1390670de338b407f623a55fc" gracePeriod=25
Oct 30 19:11:58.703010 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:58.702977    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d"
Oct 30 19:11:58.746277 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:58.746236    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57e25c31-3dbe-498f-b392-2d210ba51cd8-kube-api-access-fw4rr\" (UniqueName: \"kubernetes.io/projected/57e25c31-3dbe-498f-b392-2d210ba51cd8-kube-api-access-fw4rr\") pod \"57e25c31-3dbe-498f-b392-2d210ba51cd8\" (UID: \"57e25c31-3dbe-498f-b392-2d210ba51cd8\") "
Oct 30 19:11:58.746450 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:58.746320    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/57e25c31-3dbe-498f-b392-2d210ba51cd8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57e25c31-3dbe-498f-b392-2d210ba51cd8-plugins-config-volume\") pod \"57e25c31-3dbe-498f-b392-2d210ba51cd8\" (UID: \"57e25c31-3dbe-498f-b392-2d210ba51cd8\") "
Oct 30 19:11:58.747155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:58.747121    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57e25c31-3dbe-498f-b392-2d210ba51cd8-plugins-config-volume" pod "57e25c31-3dbe-498f-b392-2d210ba51cd8" (UID: "57e25c31-3dbe-498f-b392-2d210ba51cd8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:11:58.749019 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:58.748990    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57e25c31-3dbe-498f-b392-2d210ba51cd8-kube-api-access-fw4rr" pod "57e25c31-3dbe-498f-b392-2d210ba51cd8" (UID: "57e25c31-3dbe-498f-b392-2d210ba51cd8"). InnerVolumeSpecName "kube-api-access-fw4rr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:11:58.809723 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:58.809698    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp"
Oct 30 19:11:58.848149 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:58.848117    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04e84344-8275-4abf-b4de-61ff14cbd45c-kube-api-access-cmvk6\" (UniqueName: \"kubernetes.io/projected/04e84344-8275-4abf-b4de-61ff14cbd45c-kube-api-access-cmvk6\") pod \"04e84344-8275-4abf-b4de-61ff14cbd45c\" (UID: \"04e84344-8275-4abf-b4de-61ff14cbd45c\") "
Oct 30 19:11:58.848149 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:58.848154    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/04e84344-8275-4abf-b4de-61ff14cbd45c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04e84344-8275-4abf-b4de-61ff14cbd45c-plugins-config-volume\") pod \"04e84344-8275-4abf-b4de-61ff14cbd45c\" (UID: \"04e84344-8275-4abf-b4de-61ff14cbd45c\") "
Oct 30 19:11:58.848656 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:58.848618    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04e84344-8275-4abf-b4de-61ff14cbd45c-plugins-config-volume" pod "04e84344-8275-4abf-b4de-61ff14cbd45c" (UID: "04e84344-8275-4abf-b4de-61ff14cbd45c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:11:58.848934 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:58.848915    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04e84344-8275-4abf-b4de-61ff14cbd45c-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:11:58.849025 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:58.848936    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fw4rr\" (UniqueName: \"kubernetes.io/projected/57e25c31-3dbe-498f-b392-2d210ba51cd8-kube-api-access-fw4rr\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:11:58.849025 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:58.848946    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57e25c31-3dbe-498f-b392-2d210ba51cd8-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:11:58.850969 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:58.850941    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04e84344-8275-4abf-b4de-61ff14cbd45c-kube-api-access-cmvk6" pod "04e84344-8275-4abf-b4de-61ff14cbd45c" (UID: "04e84344-8275-4abf-b4de-61ff14cbd45c"). InnerVolumeSpecName "kube-api-access-cmvk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:11:58.950348 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:58.950264    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmvk6\" (UniqueName: \"kubernetes.io/projected/04e84344-8275-4abf-b4de-61ff14cbd45c-kube-api-access-cmvk6\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:11:59.363035 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:59.362994    4544 generic.go:356] "Generic (PLEG): container finished" podID="04e84344-8275-4abf-b4de-61ff14cbd45c" containerID="f9594d6cd8356196fae5807cbdecde5151a57ba1390670de338b407f623a55fc" exitCode=0
Oct 30 19:11:59.363243 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:59.363040    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp"
Oct 30 19:11:59.363243 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:59.363082    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp" event={"ID":"04e84344-8275-4abf-b4de-61ff14cbd45c","Type":"ContainerDied","Data":"f9594d6cd8356196fae5807cbdecde5151a57ba1390670de338b407f623a55fc"}
Oct 30 19:11:59.363243 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:59.363149    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp" event={"ID":"04e84344-8275-4abf-b4de-61ff14cbd45c","Type":"ContainerDied","Data":"9e8b98c26c0aa8d4b6b13d2c9e432ffdeb7eb63e63fda948cc81d3ef2e55fb46"}
Oct 30 19:11:59.363243 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:59.363181    4544 scope.go:122] "RemoveContainer" containerID="f9594d6cd8356196fae5807cbdecde5151a57ba1390670de338b407f623a55fc"
Oct 30 19:11:59.365018 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:59.364982    4544 generic.go:356] "Generic (PLEG): container finished" podID="57e25c31-3dbe-498f-b392-2d210ba51cd8" containerID="88c0bfd9114be0f9be7c4edb08f8a0b410850cdc210bd135430e57ca62bfc78f" exitCode=0
Oct 30 19:11:59.365105 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:59.365060    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d"
Oct 30 19:11:59.365175 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:59.365063    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d" event={"ID":"57e25c31-3dbe-498f-b392-2d210ba51cd8","Type":"ContainerDied","Data":"88c0bfd9114be0f9be7c4edb08f8a0b410850cdc210bd135430e57ca62bfc78f"}
Oct 30 19:11:59.365175 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:59.365170    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d" event={"ID":"57e25c31-3dbe-498f-b392-2d210ba51cd8","Type":"ContainerDied","Data":"9e9b8eb12b6c1cfff38dcad0ea0574293532677b2c8ffc1dccf388876496d470"}
Oct 30 19:11:59.378102 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:59.378039    4544 scope.go:122] "RemoveContainer" containerID="f9594d6cd8356196fae5807cbdecde5151a57ba1390670de338b407f623a55fc"
Oct 30 19:11:59.378444 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:11:59.378416    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9594d6cd8356196fae5807cbdecde5151a57ba1390670de338b407f623a55fc\": container with ID starting with f9594d6cd8356196fae5807cbdecde5151a57ba1390670de338b407f623a55fc not found: ID does not exist" containerID="f9594d6cd8356196fae5807cbdecde5151a57ba1390670de338b407f623a55fc"
Oct 30 19:11:59.379432 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:59.378570    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9594d6cd8356196fae5807cbdecde5151a57ba1390670de338b407f623a55fc"} err="failed to get container status \"f9594d6cd8356196fae5807cbdecde5151a57ba1390670de338b407f623a55fc\": rpc error: code = NotFound desc = could not find container \"f9594d6cd8356196fae5807cbdecde5151a57ba1390670de338b407f623a55fc\": container with ID starting with f9594d6cd8356196fae5807cbdecde5151a57ba1390670de338b407f623a55fc not found: ID does not exist"
Oct 30 19:11:59.379432 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:59.378759    4544 scope.go:122] "RemoveContainer" containerID="88c0bfd9114be0f9be7c4edb08f8a0b410850cdc210bd135430e57ca62bfc78f"
Oct 30 19:11:59.388285 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:59.388263    4544 scope.go:122] "RemoveContainer" containerID="88c0bfd9114be0f9be7c4edb08f8a0b410850cdc210bd135430e57ca62bfc78f"
Oct 30 19:11:59.388545 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:11:59.388520    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88c0bfd9114be0f9be7c4edb08f8a0b410850cdc210bd135430e57ca62bfc78f\": container with ID starting with 88c0bfd9114be0f9be7c4edb08f8a0b410850cdc210bd135430e57ca62bfc78f not found: ID does not exist" containerID="88c0bfd9114be0f9be7c4edb08f8a0b410850cdc210bd135430e57ca62bfc78f"
Oct 30 19:11:59.388642 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:59.388546    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88c0bfd9114be0f9be7c4edb08f8a0b410850cdc210bd135430e57ca62bfc78f"} err="failed to get container status \"88c0bfd9114be0f9be7c4edb08f8a0b410850cdc210bd135430e57ca62bfc78f\": rpc error: code = NotFound desc = could not find container \"88c0bfd9114be0f9be7c4edb08f8a0b410850cdc210bd135430e57ca62bfc78f\": container with ID starting with 88c0bfd9114be0f9be7c4edb08f8a0b410850cdc210bd135430e57ca62bfc78f not found: ID does not exist"
Oct 30 19:11:59.535352 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:59.535314    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d"]
Oct 30 19:11:59.542158 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:59.541828    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-071cf0-5456bc4dc8-zjn9d"]
Oct 30 19:11:59.650201 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:59.650134    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp"]
Oct 30 19:11:59.653952 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:11:59.653921    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-071cf0-854b9d8f5-h57sp"]
Oct 30 19:12:00.609625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:00.609362    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="04e84344-8275-4abf-b4de-61ff14cbd45c" path="/var/lib/kubelet/pods/04e84344-8275-4abf-b4de-61ff14cbd45c/volumes"
Oct 30 19:12:00.610059 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:00.609845    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57e25c31-3dbe-498f-b392-2d210ba51cd8" path="/var/lib/kubelet/pods/57e25c31-3dbe-498f-b392-2d210ba51cd8/volumes"
Oct 30 19:12:05.532165 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:12:05.532128    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:12:29.725992 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:29.725952    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"]
Oct 30 19:12:29.726525 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:29.726217    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm" podUID="a70cfd95-bd84-4720-ae1a-630761e61fb1" containerName="istio-proxy" containerID="cri-o://1610b2fc724d58f45c4284141b554183a2face02aa6f6f895da68b2328b87de3" gracePeriod=30
Oct 30 19:12:30.452577 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:30.452538    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"]
Oct 30 19:12:30.452788 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:30.452763    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6" podUID="f23bf845-e578-4ca7-bc75-898f940fc494" containerName="istio-proxy" containerID="cri-o://a38f71934b5adc500da74fa0dfbc2fa57110255136717e478fffedb523d18c24" gracePeriod=30
Oct 30 19:12:33.614712 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:33.614677    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:12:33.615108 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:33.614731    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6" podUID="f23bf845-e578-4ca7-bc75-898f940fc494" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:12:34.881924 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:34.881885    4544 patch_prober.go:29] interesting pod/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused" start-of-body=
Oct 30 19:12:34.882369 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:34.881954    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm" podUID="a70cfd95-bd84-4720-ae1a-630761e61fb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.106:15021/healthz/ready\": dial tcp 10.133.0.106:15021: connect: connection refused"
Oct 30 19:12:35.012430 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.012403    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:12:35.088535 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.088449    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a70cfd95-bd84-4720-ae1a-630761e61fb1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a70cfd95-bd84-4720-ae1a-630761e61fb1-istiod-ca-cert\") pod \"a70cfd95-bd84-4720-ae1a-630761e61fb1\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") "
Oct 30 19:12:35.088535 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.088499    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-envoy\") pod \"a70cfd95-bd84-4720-ae1a-630761e61fb1\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") "
Oct 30 19:12:35.088535 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.088521    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-podinfo\") pod \"a70cfd95-bd84-4720-ae1a-630761e61fb1\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") "
Oct 30 19:12:35.088849 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.088558    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-data\") pod \"a70cfd95-bd84-4720-ae1a-630761e61fb1\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") "
Oct 30 19:12:35.088849 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.088642    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a70cfd95-bd84-4720-ae1a-630761e61fb1-kube-api-access-g8ftp\" (UniqueName: \"kubernetes.io/projected/a70cfd95-bd84-4720-ae1a-630761e61fb1-kube-api-access-g8ftp\") pod \"a70cfd95-bd84-4720-ae1a-630761e61fb1\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") "
Oct 30 19:12:35.088849 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.088688    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-token\" (UniqueName: \"kubernetes.io/projected/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-token\") pod \"a70cfd95-bd84-4720-ae1a-630761e61fb1\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") "
Oct 30 19:12:35.088849 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.088810    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-credential-socket\") pod \"a70cfd95-bd84-4720-ae1a-630761e61fb1\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") "
Oct 30 19:12:35.089075 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.088873    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-workload-socket\") pod \"a70cfd95-bd84-4720-ae1a-630761e61fb1\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") "
Oct 30 19:12:35.089075 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.088918    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-workload-certs\") pod \"a70cfd95-bd84-4720-ae1a-630761e61fb1\" (UID: \"a70cfd95-bd84-4720-ae1a-630761e61fb1\") "
Oct 30 19:12:35.089075 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.088960    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-data" pod "a70cfd95-bd84-4720-ae1a-630761e61fb1" (UID: "a70cfd95-bd84-4720-ae1a-630761e61fb1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:12:35.089075 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.088992    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a70cfd95-bd84-4720-ae1a-630761e61fb1-istiod-ca-cert" pod "a70cfd95-bd84-4720-ae1a-630761e61fb1" (UID: "a70cfd95-bd84-4720-ae1a-630761e61fb1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:12:35.089299 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.089183    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-workload-socket" pod "a70cfd95-bd84-4720-ae1a-630761e61fb1" (UID: "a70cfd95-bd84-4720-ae1a-630761e61fb1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:12:35.089352 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.089291    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-credential-socket" pod "a70cfd95-bd84-4720-ae1a-630761e61fb1" (UID: "a70cfd95-bd84-4720-ae1a-630761e61fb1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:12:35.089352 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.089305    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-workload-certs" pod "a70cfd95-bd84-4720-ae1a-630761e61fb1" (UID: "a70cfd95-bd84-4720-ae1a-630761e61fb1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:12:35.089653 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.089632    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:35.089751 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.089657    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:35.089751 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.089671    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:35.089751 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.089683    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a70cfd95-bd84-4720-ae1a-630761e61fb1-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:35.089751 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.089696    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:35.091006 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.090979    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-envoy" pod "a70cfd95-bd84-4720-ae1a-630761e61fb1" (UID: "a70cfd95-bd84-4720-ae1a-630761e61fb1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:12:35.091006 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.090991    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-podinfo" pod "a70cfd95-bd84-4720-ae1a-630761e61fb1" (UID: "a70cfd95-bd84-4720-ae1a-630761e61fb1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:12:35.091245 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.091225    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-token" pod "a70cfd95-bd84-4720-ae1a-630761e61fb1" (UID: "a70cfd95-bd84-4720-ae1a-630761e61fb1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:12:35.091410 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.091393    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a70cfd95-bd84-4720-ae1a-630761e61fb1-kube-api-access-g8ftp" pod "a70cfd95-bd84-4720-ae1a-630761e61fb1" (UID: "a70cfd95-bd84-4720-ae1a-630761e61fb1"). InnerVolumeSpecName "kube-api-access-g8ftp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:12:35.191560 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.191514    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:35.191560 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.191564    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:35.191776 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.191580    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8ftp\" (UniqueName: \"kubernetes.io/projected/a70cfd95-bd84-4720-ae1a-630761e61fb1-kube-api-access-g8ftp\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:35.191776 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.191594    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a70cfd95-bd84-4720-ae1a-630761e61fb1-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:35.533112 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:12:35.533076    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:12:35.674271 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.674239    4544 generic.go:356] "Generic (PLEG): container finished" podID="f23bf845-e578-4ca7-bc75-898f940fc494" containerID="a38f71934b5adc500da74fa0dfbc2fa57110255136717e478fffedb523d18c24" exitCode=0
Oct 30 19:12:35.674466 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.674309    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6" event={"ID":"f23bf845-e578-4ca7-bc75-898f940fc494","Type":"ContainerDied","Data":"a38f71934b5adc500da74fa0dfbc2fa57110255136717e478fffedb523d18c24"}
Oct 30 19:12:35.676725 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.676700    4544 generic.go:356] "Generic (PLEG): container finished" podID="a70cfd95-bd84-4720-ae1a-630761e61fb1" containerID="1610b2fc724d58f45c4284141b554183a2face02aa6f6f895da68b2328b87de3" exitCode=0
Oct 30 19:12:35.676867 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.676783    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm" event={"ID":"a70cfd95-bd84-4720-ae1a-630761e61fb1","Type":"ContainerDied","Data":"1610b2fc724d58f45c4284141b554183a2face02aa6f6f895da68b2328b87de3"}
Oct 30 19:12:35.676867 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.676823    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm" event={"ID":"a70cfd95-bd84-4720-ae1a-630761e61fb1","Type":"ContainerDied","Data":"7bfd7a3c8c5d0a688335638c8f9e0d9e1651c1a802849d845d1223b2939ae90b"}
Oct 30 19:12:35.676867 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.676845    4544 scope.go:122] "RemoveContainer" containerID="1610b2fc724d58f45c4284141b554183a2face02aa6f6f895da68b2328b87de3"
Oct 30 19:12:35.677038 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.676785    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"
Oct 30 19:12:35.692490 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.692468    4544 scope.go:122] "RemoveContainer" containerID="5858631fe9c06128edf98eb170e84e080c068edc31951644d87a67d6f64fe3e8"
Oct 30 19:12:35.704176 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.704154    4544 scope.go:122] "RemoveContainer" containerID="1610b2fc724d58f45c4284141b554183a2face02aa6f6f895da68b2328b87de3"
Oct 30 19:12:35.704505 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:12:35.704478    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1610b2fc724d58f45c4284141b554183a2face02aa6f6f895da68b2328b87de3\": container with ID starting with 1610b2fc724d58f45c4284141b554183a2face02aa6f6f895da68b2328b87de3 not found: ID does not exist" containerID="1610b2fc724d58f45c4284141b554183a2face02aa6f6f895da68b2328b87de3"
Oct 30 19:12:35.704592 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.704517    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1610b2fc724d58f45c4284141b554183a2face02aa6f6f895da68b2328b87de3"} err="failed to get container status \"1610b2fc724d58f45c4284141b554183a2face02aa6f6f895da68b2328b87de3\": rpc error: code = NotFound desc = could not find container \"1610b2fc724d58f45c4284141b554183a2face02aa6f6f895da68b2328b87de3\": container with ID starting with 1610b2fc724d58f45c4284141b554183a2face02aa6f6f895da68b2328b87de3 not found: ID does not exist"
Oct 30 19:12:35.704592 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.704545    4544 scope.go:122] "RemoveContainer" containerID="5858631fe9c06128edf98eb170e84e080c068edc31951644d87a67d6f64fe3e8"
Oct 30 19:12:35.705346 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:12:35.705322    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5858631fe9c06128edf98eb170e84e080c068edc31951644d87a67d6f64fe3e8\": container with ID starting with 5858631fe9c06128edf98eb170e84e080c068edc31951644d87a67d6f64fe3e8 not found: ID does not exist" containerID="5858631fe9c06128edf98eb170e84e080c068edc31951644d87a67d6f64fe3e8"
Oct 30 19:12:35.705422 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.705355    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5858631fe9c06128edf98eb170e84e080c068edc31951644d87a67d6f64fe3e8"} err="failed to get container status \"5858631fe9c06128edf98eb170e84e080c068edc31951644d87a67d6f64fe3e8\": rpc error: code = NotFound desc = could not find container \"5858631fe9c06128edf98eb170e84e080c068edc31951644d87a67d6f64fe3e8\": container with ID starting with 5858631fe9c06128edf98eb170e84e080c068edc31951644d87a67d6f64fe3e8 not found: ID does not exist"
Oct 30 19:12:35.716218 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.716198    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 19:12:35.806502 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.806465    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f23bf845-e578-4ca7-bc75-898f940fc494-istio-token\" (UniqueName: \"kubernetes.io/projected/f23bf845-e578-4ca7-bc75-898f940fc494-istio-token\") pod \"f23bf845-e578-4ca7-bc75-898f940fc494\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") "
Oct 30 19:12:35.806722 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.806535    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f23bf845-e578-4ca7-bc75-898f940fc494-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f23bf845-e578-4ca7-bc75-898f940fc494-istio-podinfo\") pod \"f23bf845-e578-4ca7-bc75-898f940fc494\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") "
Oct 30 19:12:35.806722 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.806588    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f23bf845-e578-4ca7-bc75-898f940fc494-kube-api-access-7kjb5\" (UniqueName: \"kubernetes.io/projected/f23bf845-e578-4ca7-bc75-898f940fc494-kube-api-access-7kjb5\") pod \"f23bf845-e578-4ca7-bc75-898f940fc494\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") "
Oct 30 19:12:35.806722 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.806642    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f23bf845-e578-4ca7-bc75-898f940fc494-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f23bf845-e578-4ca7-bc75-898f940fc494-istiod-ca-cert\") pod \"f23bf845-e578-4ca7-bc75-898f940fc494\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") "
Oct 30 19:12:35.806866 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.806778    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-workload-certs\") pod \"f23bf845-e578-4ca7-bc75-898f940fc494\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") "
Oct 30 19:12:35.806866 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.806817    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-istio-data\") pod \"f23bf845-e578-4ca7-bc75-898f940fc494\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") "
Oct 30 19:12:35.806969 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.806902    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-istio-envoy\") pod \"f23bf845-e578-4ca7-bc75-898f940fc494\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") "
Oct 30 19:12:35.806969 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.806940    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-credential-socket\") pod \"f23bf845-e578-4ca7-bc75-898f940fc494\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") "
Oct 30 19:12:35.807068 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.807031    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-workload-socket\") pod \"f23bf845-e578-4ca7-bc75-898f940fc494\" (UID: \"f23bf845-e578-4ca7-bc75-898f940fc494\") "
Oct 30 19:12:35.807175 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.807147    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f23bf845-e578-4ca7-bc75-898f940fc494-istiod-ca-cert" pod "f23bf845-e578-4ca7-bc75-898f940fc494" (UID: "f23bf845-e578-4ca7-bc75-898f940fc494"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:12:35.807339 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.807318    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-workload-certs" pod "f23bf845-e578-4ca7-bc75-898f940fc494" (UID: "f23bf845-e578-4ca7-bc75-898f940fc494"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:12:35.807546 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.807519    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-istio-data" pod "f23bf845-e578-4ca7-bc75-898f940fc494" (UID: "f23bf845-e578-4ca7-bc75-898f940fc494"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:12:35.807708 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.807687    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f23bf845-e578-4ca7-bc75-898f940fc494-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:35.807802 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.807714    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:35.807802 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.807740    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-credential-socket" pod "f23bf845-e578-4ca7-bc75-898f940fc494" (UID: "f23bf845-e578-4ca7-bc75-898f940fc494"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:12:35.808093 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.807534    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-workload-socket" pod "f23bf845-e578-4ca7-bc75-898f940fc494" (UID: "f23bf845-e578-4ca7-bc75-898f940fc494"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:12:35.809919 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.809890    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-istio-envoy" pod "f23bf845-e578-4ca7-bc75-898f940fc494" (UID: "f23bf845-e578-4ca7-bc75-898f940fc494"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:12:35.810022 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.809923    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f23bf845-e578-4ca7-bc75-898f940fc494-kube-api-access-7kjb5" pod "f23bf845-e578-4ca7-bc75-898f940fc494" (UID: "f23bf845-e578-4ca7-bc75-898f940fc494"). InnerVolumeSpecName "kube-api-access-7kjb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:12:35.810022 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.809955    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f23bf845-e578-4ca7-bc75-898f940fc494-istio-token" pod "f23bf845-e578-4ca7-bc75-898f940fc494" (UID: "f23bf845-e578-4ca7-bc75-898f940fc494"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:12:35.810703 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.810685    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f23bf845-e578-4ca7-bc75-898f940fc494-istio-podinfo" pod "f23bf845-e578-4ca7-bc75-898f940fc494" (UID: "f23bf845-e578-4ca7-bc75-898f940fc494"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:12:35.909285 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.909248    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:35.909285 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.909279    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:35.909285 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.909290    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:35.909895 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.909299    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f23bf845-e578-4ca7-bc75-898f940fc494-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:35.909895 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.909308    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f23bf845-e578-4ca7-bc75-898f940fc494-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:35.909895 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.909317    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kjb5\" (UniqueName: \"kubernetes.io/projected/f23bf845-e578-4ca7-bc75-898f940fc494-kube-api-access-7kjb5\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:35.909895 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.909328    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f23bf845-e578-4ca7-bc75-898f940fc494-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:35.964976 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.964938    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"]
Oct 30 19:12:35.972246 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:35.972212    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-cfb433-openshift-default-6ddc44dc55-httgm"]
Oct 30 19:12:36.581947 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:36.581912    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"]
Oct 30 19:12:36.582230 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:36.582201    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr" podUID="b1badcf1-f26a-4a5e-a068-678c9971e21c" containerName="istio-proxy" containerID="cri-o://4a12a3cc0f0de8c7798f0a3039d44161999c009dba82d1c1881e9a43711833af" gracePeriod=30
Oct 30 19:12:36.603424 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:36.603390    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a70cfd95-bd84-4720-ae1a-630761e61fb1" path="/var/lib/kubelet/pods/a70cfd95-bd84-4720-ae1a-630761e61fb1/volumes"
Oct 30 19:12:36.771481 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:36.768835    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6" event={"ID":"f23bf845-e578-4ca7-bc75-898f940fc494","Type":"ContainerDied","Data":"8c26acdec99ecb6dcec4ea1d03abfa4a61c5d85a824a60e354783b7be315300d"}
Oct 30 19:12:36.771481 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:36.768909    4544 scope.go:122] "RemoveContainer" containerID="a38f71934b5adc500da74fa0dfbc2fa57110255136717e478fffedb523d18c24"
Oct 30 19:12:36.771481 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:36.769063    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"
Oct 30 19:12:36.939648 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:36.939558    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"]
Oct 30 19:12:36.944581 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:36.944558    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-gbtx6"]
Oct 30 19:12:38.605645 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:38.605595    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f23bf845-e578-4ca7-bc75-898f940fc494" path="/var/lib/kubelet/pods/f23bf845-e578-4ca7-bc75-898f940fc494/volumes"
Oct 30 19:12:41.600459 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.600424    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq"]
Oct 30 19:12:41.601287 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601260    4544 cpu_manager.go:414] "containerMap: removing container" podUID="50b8d8f8-30ae-4c27-9e2f-44bd646c916d" containerName="payload-processing"
Oct 30 19:12:41.601287 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601283    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50b8d8f8-30ae-4c27-9e2f-44bd646c916d" containerName="payload-processing"
Oct 30 19:12:41.601462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601305    4544 cpu_manager.go:414] "containerMap: removing container" podUID="4f494989-34da-4a63-95c2-7440b320bb0a" containerName="istio-proxy"
Oct 30 19:12:41.601462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601313    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4f494989-34da-4a63-95c2-7440b320bb0a" containerName="istio-proxy"
Oct 30 19:12:41.601462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601326    4544 cpu_manager.go:414] "containerMap: removing container" podUID="8955fdbb-df1e-4a72-83c2-bef3dbb86a68" containerName="payload-pre-processing"
Oct 30 19:12:41.601462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601333    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8955fdbb-df1e-4a72-83c2-bef3dbb86a68" containerName="payload-pre-processing"
Oct 30 19:12:41.601462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601362    4544 cpu_manager.go:414] "containerMap: removing container" podUID="76db530e-e95b-498c-be91-e4bb9291fa43" containerName="payload-pre-processing"
Oct 30 19:12:41.601462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601369    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76db530e-e95b-498c-be91-e4bb9291fa43" containerName="payload-pre-processing"
Oct 30 19:12:41.601462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601383    4544 cpu_manager.go:414] "containerMap: removing container" podUID="04e84344-8275-4abf-b4de-61ff14cbd45c" containerName="payload-pre-processing"
Oct 30 19:12:41.601462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601390    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04e84344-8275-4abf-b4de-61ff14cbd45c" containerName="payload-pre-processing"
Oct 30 19:12:41.601462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601404    4544 cpu_manager.go:414] "containerMap: removing container" podUID="f23bf845-e578-4ca7-bc75-898f940fc494" containerName="istio-proxy"
Oct 30 19:12:41.601462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601410    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f23bf845-e578-4ca7-bc75-898f940fc494" containerName="istio-proxy"
Oct 30 19:12:41.601462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601445    4544 cpu_manager.go:414] "containerMap: removing container" podUID="a70cfd95-bd84-4720-ae1a-630761e61fb1" containerName="istio-proxy"
Oct 30 19:12:41.601462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601452    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a70cfd95-bd84-4720-ae1a-630761e61fb1" containerName="istio-proxy"
Oct 30 19:12:41.601462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601462    4544 cpu_manager.go:414] "containerMap: removing container" podUID="1f93fb78-4ca3-4012-8d92-9f98d8747614" containerName="payload-pre-processing"
Oct 30 19:12:41.602155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601470    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f93fb78-4ca3-4012-8d92-9f98d8747614" containerName="payload-pre-processing"
Oct 30 19:12:41.602155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601482    4544 cpu_manager.go:414] "containerMap: removing container" podUID="a70cfd95-bd84-4720-ae1a-630761e61fb1" containerName="istio-proxy"
Oct 30 19:12:41.602155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601492    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a70cfd95-bd84-4720-ae1a-630761e61fb1" containerName="istio-proxy"
Oct 30 19:12:41.602155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601504    4544 cpu_manager.go:414] "containerMap: removing container" podUID="9877e03e-92b6-43c7-83c7-6fdea1d04000" containerName="payload-processing"
Oct 30 19:12:41.602155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601511    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9877e03e-92b6-43c7-83c7-6fdea1d04000" containerName="payload-processing"
Oct 30 19:12:41.602155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601532    4544 cpu_manager.go:414] "containerMap: removing container" podUID="57e25c31-3dbe-498f-b392-2d210ba51cd8" containerName="payload-processing"
Oct 30 19:12:41.602155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601539    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57e25c31-3dbe-498f-b392-2d210ba51cd8" containerName="payload-processing"
Oct 30 19:12:41.602155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601710    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="f23bf845-e578-4ca7-bc75-898f940fc494" containerName="istio-proxy"
Oct 30 19:12:41.602155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601729    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="a70cfd95-bd84-4720-ae1a-630761e61fb1" containerName="istio-proxy"
Oct 30 19:12:41.602155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601740    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f93fb78-4ca3-4012-8d92-9f98d8747614" containerName="payload-pre-processing"
Oct 30 19:12:41.602155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601749    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="a70cfd95-bd84-4720-ae1a-630761e61fb1" containerName="istio-proxy"
Oct 30 19:12:41.602155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601757    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="9877e03e-92b6-43c7-83c7-6fdea1d04000" containerName="payload-processing"
Oct 30 19:12:41.602155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601769    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="57e25c31-3dbe-498f-b392-2d210ba51cd8" containerName="payload-processing"
Oct 30 19:12:41.602155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601780    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="50b8d8f8-30ae-4c27-9e2f-44bd646c916d" containerName="payload-processing"
Oct 30 19:12:41.602155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601791    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="4f494989-34da-4a63-95c2-7440b320bb0a" containerName="istio-proxy"
Oct 30 19:12:41.602155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601800    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="8955fdbb-df1e-4a72-83c2-bef3dbb86a68" containerName="payload-pre-processing"
Oct 30 19:12:41.602155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601814    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="76db530e-e95b-498c-be91-e4bb9291fa43" containerName="payload-pre-processing"
Oct 30 19:12:41.602155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.601825    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="04e84344-8275-4abf-b4de-61ff14cbd45c" containerName="payload-pre-processing"
Oct 30 19:12:41.605416 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.605395    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq"
Oct 30 19:12:41.670798 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.670763    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq"]
Oct 30 19:12:41.714634 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.708193    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-f75d5f\""
Oct 30 19:12:41.714634 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.708347    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-f75d5f-dockercfg-6hn9v\""
Oct 30 19:12:41.741626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.726244    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrhzq\" (UniqueName: \"kubernetes.io/projected/8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3-kube-api-access-rrhzq\") pod \"payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq\" (UID: \"8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq"
Oct 30 19:12:41.741626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.726459    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq\" (UID: \"8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq"
Oct 30 19:12:41.830065 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.830024    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq\" (UID: \"8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq"
Oct 30 19:12:41.830234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.830174    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrhzq\" (UniqueName: \"kubernetes.io/projected/8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3-kube-api-access-rrhzq\") pod \"payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq\" (UID: \"8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq"
Oct 30 19:12:41.831520 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.831478    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq\" (UID: \"8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq"
Oct 30 19:12:41.843655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.840833    4544 generic.go:356] "Generic (PLEG): container finished" podID="b1badcf1-f26a-4a5e-a068-678c9971e21c" containerID="4a12a3cc0f0de8c7798f0a3039d44161999c009dba82d1c1881e9a43711833af" exitCode=0
Oct 30 19:12:41.843655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.841049    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr" event={"ID":"b1badcf1-f26a-4a5e-a068-678c9971e21c","Type":"ContainerDied","Data":"4a12a3cc0f0de8c7798f0a3039d44161999c009dba82d1c1881e9a43711833af"}
Oct 30 19:12:41.843655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.841096    4544 scope.go:122] "RemoveContainer" containerID="33e8b84fc9d65eb9bbcb59ec7f1ead7c6a97f19e1594a6eb8517062a87c74171"
Oct 30 19:12:41.904369 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:41.902575    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrhzq\" (UniqueName: \"kubernetes.io/projected/8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3-kube-api-access-rrhzq\") pod \"payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq\" (UID: \"8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq"
Oct 30 19:12:42.011671 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.011639    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:12:42.031542 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.031512    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq"
Oct 30 19:12:42.134311 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.134162    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-workload-certs\") pod \"b1badcf1-f26a-4a5e-a068-678c9971e21c\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") "
Oct 30 19:12:42.134311 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.134288    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-envoy\") pod \"b1badcf1-f26a-4a5e-a068-678c9971e21c\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") "
Oct 30 19:12:42.134586 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.134371    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-credential-socket\") pod \"b1badcf1-f26a-4a5e-a068-678c9971e21c\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") "
Oct 30 19:12:42.134586 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.134529    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-podinfo\") pod \"b1badcf1-f26a-4a5e-a068-678c9971e21c\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") "
Oct 30 19:12:42.134586 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.134568    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-token\" (UniqueName: \"kubernetes.io/projected/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-token\") pod \"b1badcf1-f26a-4a5e-a068-678c9971e21c\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") "
Oct 30 19:12:42.134889 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.134694    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1badcf1-f26a-4a5e-a068-678c9971e21c-kube-api-access-p5xsw\" (UniqueName: \"kubernetes.io/projected/b1badcf1-f26a-4a5e-a068-678c9971e21c-kube-api-access-p5xsw\") pod \"b1badcf1-f26a-4a5e-a068-678c9971e21c\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") "
Oct 30 19:12:42.134889 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.134727    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1badcf1-f26a-4a5e-a068-678c9971e21c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1badcf1-f26a-4a5e-a068-678c9971e21c-istiod-ca-cert\") pod \"b1badcf1-f26a-4a5e-a068-678c9971e21c\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") "
Oct 30 19:12:42.134889 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.134808    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-workload-socket\") pod \"b1badcf1-f26a-4a5e-a068-678c9971e21c\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") "
Oct 30 19:12:42.135110 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.134913    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-data\") pod \"b1badcf1-f26a-4a5e-a068-678c9971e21c\" (UID: \"b1badcf1-f26a-4a5e-a068-678c9971e21c\") "
Oct 30 19:12:42.135983 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.135933    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-data" pod "b1badcf1-f26a-4a5e-a068-678c9971e21c" (UID: "b1badcf1-f26a-4a5e-a068-678c9971e21c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:12:42.137648 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.136285    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1badcf1-f26a-4a5e-a068-678c9971e21c-istiod-ca-cert" pod "b1badcf1-f26a-4a5e-a068-678c9971e21c" (UID: "b1badcf1-f26a-4a5e-a068-678c9971e21c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:12:42.137648 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.136538    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-workload-socket" pod "b1badcf1-f26a-4a5e-a068-678c9971e21c" (UID: "b1badcf1-f26a-4a5e-a068-678c9971e21c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:12:42.137648 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.136784    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-workload-certs" pod "b1badcf1-f26a-4a5e-a068-678c9971e21c" (UID: "b1badcf1-f26a-4a5e-a068-678c9971e21c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:12:42.137648 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.136840    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-credential-socket" pod "b1badcf1-f26a-4a5e-a068-678c9971e21c" (UID: "b1badcf1-f26a-4a5e-a068-678c9971e21c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:12:42.142471 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.142129    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1badcf1-f26a-4a5e-a068-678c9971e21c-kube-api-access-p5xsw" pod "b1badcf1-f26a-4a5e-a068-678c9971e21c" (UID: "b1badcf1-f26a-4a5e-a068-678c9971e21c"). InnerVolumeSpecName "kube-api-access-p5xsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:12:42.142994 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.142972    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-podinfo" pod "b1badcf1-f26a-4a5e-a068-678c9971e21c" (UID: "b1badcf1-f26a-4a5e-a068-678c9971e21c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:12:42.143089 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.142977    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-envoy" pod "b1badcf1-f26a-4a5e-a068-678c9971e21c" (UID: "b1badcf1-f26a-4a5e-a068-678c9971e21c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:12:42.143694 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.143674    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-token" pod "b1badcf1-f26a-4a5e-a068-678c9971e21c" (UID: "b1badcf1-f26a-4a5e-a068-678c9971e21c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:12:42.236490 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.236447    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:42.236490 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.236481    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:42.236490 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.236495    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:42.236785 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.236508    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:42.236785 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.236520    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:42.236785 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.236531    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1badcf1-f26a-4a5e-a068-678c9971e21c-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:42.236785 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.236542    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5xsw\" (UniqueName: \"kubernetes.io/projected/b1badcf1-f26a-4a5e-a068-678c9971e21c-kube-api-access-p5xsw\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:42.236785 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.236555    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1badcf1-f26a-4a5e-a068-678c9971e21c-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:42.236785 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.236568    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1badcf1-f26a-4a5e-a068-678c9971e21c-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:42.416410 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:12:42.416368    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e9121ca_ec5a_4c12_a0e7_f9b70d561ef3.slice/crio-1beac4d83aeffd4938b67ff00f1019b4015a763c40fa779f46f34342d240462c WatchSource:0}: Error finding container 1beac4d83aeffd4938b67ff00f1019b4015a763c40fa779f46f34342d240462c: Status 404 returned error can't find the container with id 1beac4d83aeffd4938b67ff00f1019b4015a763c40fa779f46f34342d240462c
Oct 30 19:12:42.419253 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.419231    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq"]
Oct 30 19:12:42.849141 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.849109    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq" event={"ID":"8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3","Type":"ContainerStarted","Data":"2beb9dd333097b62ffa5094802ee984b37f78e57b87df22d8fdaf841f8481d89"}
Oct 30 19:12:42.849570 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.849159    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq" event={"ID":"8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3","Type":"ContainerStarted","Data":"1beac4d83aeffd4938b67ff00f1019b4015a763c40fa779f46f34342d240462c"}
Oct 30 19:12:42.849570 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.849204    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq"
Oct 30 19:12:42.850552 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.850533    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"
Oct 30 19:12:42.850673 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.850577    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr" event={"ID":"b1badcf1-f26a-4a5e-a068-678c9971e21c","Type":"ContainerDied","Data":"175f6d9ea51e3a93531a0460dc9ca812a59b07b248ce3cd6d73a328ed862630b"}
Oct 30 19:12:42.850673 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.850640    4544 scope.go:122] "RemoveContainer" containerID="4a12a3cc0f0de8c7798f0a3039d44161999c009dba82d1c1881e9a43711833af"
Oct 30 19:12:42.977220 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:42.977170    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq" podStartSLOduration=1.977155848 podStartE2EDuration="1.977155848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:42.976103645 +0000 UTC m=+2160.501978743" watchObservedRunningTime="2026-10-30 19:12:42.977155848 +0000 UTC m=+2160.503030944"
Oct 30 19:12:43.103677 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:43.102972    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"]
Oct 30 19:12:43.148064 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:43.148025    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-cc079d-openshift-default-7679b6bfc9-2rtpr"]
Oct 30 19:12:44.605938 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:44.605901    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1badcf1-f26a-4a5e-a068-678c9971e21c" path="/var/lib/kubelet/pods/b1badcf1-f26a-4a5e-a068-678c9971e21c/volumes"
Oct 30 19:12:53.863201 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:53.863167    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq"
Oct 30 19:12:56.432357 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:56.432330    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h"]
Oct 30 19:12:56.542843 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:12:56.542806    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r"]
Oct 30 19:13:01.432797 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:01.432755    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h" podUID="f117897d-7353-4281-909c-a475e208036d" containerName="payload-processing" containerID="cri-o://c0cad08639fbc527ba4edcc5f52b4d6e375199290ac9102c468d5a41d6fd75b0" gracePeriod=25
Oct 30 19:13:01.543356 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:01.543318    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r" podUID="7482faa1-c2d0-4c2d-8998-b71343b0a141" containerName="payload-pre-processing" containerID="cri-o://b2dc66d236f58b649a8ed687354cffaabd406d80e8355d29be34b4df30cf6bc3" gracePeriod=25
Oct 30 19:13:01.725798 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:01.725750    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h"
Oct 30 19:13:01.795626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:01.794830    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f117897d-7353-4281-909c-a475e208036d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f117897d-7353-4281-909c-a475e208036d-plugins-config-volume\") pod \"f117897d-7353-4281-909c-a475e208036d\" (UID: \"f117897d-7353-4281-909c-a475e208036d\") "
Oct 30 19:13:01.795626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:01.795184    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f117897d-7353-4281-909c-a475e208036d-kube-api-access-fmsb5\" (UniqueName: \"kubernetes.io/projected/f117897d-7353-4281-909c-a475e208036d-kube-api-access-fmsb5\") pod \"f117897d-7353-4281-909c-a475e208036d\" (UID: \"f117897d-7353-4281-909c-a475e208036d\") "
Oct 30 19:13:01.795626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:01.795297    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f117897d-7353-4281-909c-a475e208036d-plugins-config-volume" pod "f117897d-7353-4281-909c-a475e208036d" (UID: "f117897d-7353-4281-909c-a475e208036d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:01.795934 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:01.795818    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f117897d-7353-4281-909c-a475e208036d-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:01.800630 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:01.800383    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f117897d-7353-4281-909c-a475e208036d-kube-api-access-fmsb5" pod "f117897d-7353-4281-909c-a475e208036d" (UID: "f117897d-7353-4281-909c-a475e208036d"). InnerVolumeSpecName "kube-api-access-fmsb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:01.873105 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:01.873073    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r"
Oct 30 19:13:01.897461 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:01.897416    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmsb5\" (UniqueName: \"kubernetes.io/projected/f117897d-7353-4281-909c-a475e208036d-kube-api-access-fmsb5\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:02.000620 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.000520    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7482faa1-c2d0-4c2d-8998-b71343b0a141-kube-api-access-s54tx\" (UniqueName: \"kubernetes.io/projected/7482faa1-c2d0-4c2d-8998-b71343b0a141-kube-api-access-s54tx\") pod \"7482faa1-c2d0-4c2d-8998-b71343b0a141\" (UID: \"7482faa1-c2d0-4c2d-8998-b71343b0a141\") "
Oct 30 19:13:02.000805 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.000648    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7482faa1-c2d0-4c2d-8998-b71343b0a141-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7482faa1-c2d0-4c2d-8998-b71343b0a141-plugins-config-volume\") pod \"7482faa1-c2d0-4c2d-8998-b71343b0a141\" (UID: \"7482faa1-c2d0-4c2d-8998-b71343b0a141\") "
Oct 30 19:13:02.001024 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.001000    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7482faa1-c2d0-4c2d-8998-b71343b0a141-plugins-config-volume" pod "7482faa1-c2d0-4c2d-8998-b71343b0a141" (UID: "7482faa1-c2d0-4c2d-8998-b71343b0a141"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:02.001337 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.001319    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7482faa1-c2d0-4c2d-8998-b71343b0a141-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:02.003484 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.003455    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7482faa1-c2d0-4c2d-8998-b71343b0a141-kube-api-access-s54tx" pod "7482faa1-c2d0-4c2d-8998-b71343b0a141" (UID: "7482faa1-c2d0-4c2d-8998-b71343b0a141"). InnerVolumeSpecName "kube-api-access-s54tx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:02.007754 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.007724    4544 generic.go:356] "Generic (PLEG): container finished" podID="7482faa1-c2d0-4c2d-8998-b71343b0a141" containerID="b2dc66d236f58b649a8ed687354cffaabd406d80e8355d29be34b4df30cf6bc3" exitCode=0
Oct 30 19:13:02.007866 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.007776    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r"
Oct 30 19:13:02.007866 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.007829    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r" event={"ID":"7482faa1-c2d0-4c2d-8998-b71343b0a141","Type":"ContainerDied","Data":"b2dc66d236f58b649a8ed687354cffaabd406d80e8355d29be34b4df30cf6bc3"}
Oct 30 19:13:02.007984 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.007875    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r" event={"ID":"7482faa1-c2d0-4c2d-8998-b71343b0a141","Type":"ContainerDied","Data":"8c4f3c519007d1a37cacbd41e64518a79335cc6fb5a31e89ca545591661e6837"}
Oct 30 19:13:02.007984 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.007902    4544 scope.go:122] "RemoveContainer" containerID="b2dc66d236f58b649a8ed687354cffaabd406d80e8355d29be34b4df30cf6bc3"
Oct 30 19:13:02.010359 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.010330    4544 generic.go:356] "Generic (PLEG): container finished" podID="f117897d-7353-4281-909c-a475e208036d" containerID="c0cad08639fbc527ba4edcc5f52b4d6e375199290ac9102c468d5a41d6fd75b0" exitCode=0
Oct 30 19:13:02.010475 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.010382    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h"
Oct 30 19:13:02.010475 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.010388    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h" event={"ID":"f117897d-7353-4281-909c-a475e208036d","Type":"ContainerDied","Data":"c0cad08639fbc527ba4edcc5f52b4d6e375199290ac9102c468d5a41d6fd75b0"}
Oct 30 19:13:02.010475 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.010457    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h" event={"ID":"f117897d-7353-4281-909c-a475e208036d","Type":"ContainerDied","Data":"2c4db7bfd832fc34e1bb7fcee3a5efb688d8d6bc502c033e14ffe0a074077373"}
Oct 30 19:13:02.020841 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.020771    4544 scope.go:122] "RemoveContainer" containerID="b2dc66d236f58b649a8ed687354cffaabd406d80e8355d29be34b4df30cf6bc3"
Oct 30 19:13:02.021207 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:13:02.021185    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2dc66d236f58b649a8ed687354cffaabd406d80e8355d29be34b4df30cf6bc3\": container with ID starting with b2dc66d236f58b649a8ed687354cffaabd406d80e8355d29be34b4df30cf6bc3 not found: ID does not exist" containerID="b2dc66d236f58b649a8ed687354cffaabd406d80e8355d29be34b4df30cf6bc3"
Oct 30 19:13:02.021302 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.021212    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2dc66d236f58b649a8ed687354cffaabd406d80e8355d29be34b4df30cf6bc3"} err="failed to get container status \"b2dc66d236f58b649a8ed687354cffaabd406d80e8355d29be34b4df30cf6bc3\": rpc error: code = NotFound desc = could not find container \"b2dc66d236f58b649a8ed687354cffaabd406d80e8355d29be34b4df30cf6bc3\": container with ID starting with b2dc66d236f58b649a8ed687354cffaabd406d80e8355d29be34b4df30cf6bc3 not found: ID does not exist"
Oct 30 19:13:02.021302 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.021231    4544 scope.go:122] "RemoveContainer" containerID="c0cad08639fbc527ba4edcc5f52b4d6e375199290ac9102c468d5a41d6fd75b0"
Oct 30 19:13:02.032665 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.032644    4544 scope.go:122] "RemoveContainer" containerID="c0cad08639fbc527ba4edcc5f52b4d6e375199290ac9102c468d5a41d6fd75b0"
Oct 30 19:13:02.032950 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:13:02.032931    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0cad08639fbc527ba4edcc5f52b4d6e375199290ac9102c468d5a41d6fd75b0\": container with ID starting with c0cad08639fbc527ba4edcc5f52b4d6e375199290ac9102c468d5a41d6fd75b0 not found: ID does not exist" containerID="c0cad08639fbc527ba4edcc5f52b4d6e375199290ac9102c468d5a41d6fd75b0"
Oct 30 19:13:02.033027 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.032955    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0cad08639fbc527ba4edcc5f52b4d6e375199290ac9102c468d5a41d6fd75b0"} err="failed to get container status \"c0cad08639fbc527ba4edcc5f52b4d6e375199290ac9102c468d5a41d6fd75b0\": rpc error: code = NotFound desc = could not find container \"c0cad08639fbc527ba4edcc5f52b4d6e375199290ac9102c468d5a41d6fd75b0\": container with ID starting with c0cad08639fbc527ba4edcc5f52b4d6e375199290ac9102c468d5a41d6fd75b0 not found: ID does not exist"
Oct 30 19:13:02.103148 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.103105    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s54tx\" (UniqueName: \"kubernetes.io/projected/7482faa1-c2d0-4c2d-8998-b71343b0a141-kube-api-access-s54tx\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:02.194769 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.194731    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r"]
Oct 30 19:13:02.203665 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.203597    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-05565d-548869fbff-78j5r"]
Oct 30 19:13:02.330368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.330336    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h"]
Oct 30 19:13:02.342181 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.342151    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-05565d-64bfd9f668-dfj2h"]
Oct 30 19:13:02.605497 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.605411    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7482faa1-c2d0-4c2d-8998-b71343b0a141" path="/var/lib/kubelet/pods/7482faa1-c2d0-4c2d-8998-b71343b0a141/volumes"
Oct 30 19:13:02.605991 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:02.605972    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f117897d-7353-4281-909c-a475e208036d" path="/var/lib/kubelet/pods/f117897d-7353-4281-909c-a475e208036d/volumes"
Oct 30 19:13:05.531640 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:13:05.531578    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:13:15.802390 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:15.802351    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"]
Oct 30 19:13:15.803073 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:15.802927    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn" podUID="b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" containerName="istio-proxy" containerID="cri-o://14e7557314a12db33fe80d6446599c175210ba979d5778b68d022f52a3aab00b" gracePeriod=30
Oct 30 19:13:18.277966 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:18.277928    4544 patch_prober.go:29] interesting pod/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:13:18.278422 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:18.278002    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn" podUID="b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:13:19.709183 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:19.709148    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz"]
Oct 30 19:13:19.814818 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:19.814787    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl"]
Oct 30 19:13:21.241523 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.241487    4544 generic.go:356] "Generic (PLEG): container finished" podID="b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" containerID="14e7557314a12db33fe80d6446599c175210ba979d5778b68d022f52a3aab00b" exitCode=0
Oct 30 19:13:21.241972 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.241574    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn" event={"ID":"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4","Type":"ContainerDied","Data":"14e7557314a12db33fe80d6446599c175210ba979d5778b68d022f52a3aab00b"}
Oct 30 19:13:21.279116 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.279082    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 19:13:21.338360 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.338311    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-workload-socket\") pod \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") "
Oct 30 19:13:21.338577 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.338546    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-workload-socket" pod "b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" (UID: "b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:21.338728 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.338651    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-envoy\") pod \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") "
Oct 30 19:13:21.338809 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.338777    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-workload-certs\") pod \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") "
Oct 30 19:13:21.338872 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.338824    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-kube-api-access-xq7rb\" (UniqueName: \"kubernetes.io/projected/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-kube-api-access-xq7rb\") pod \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") "
Oct 30 19:13:21.338931 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.338875    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-podinfo\") pod \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") "
Oct 30 19:13:21.339006 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.338988    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-data\") pod \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") "
Oct 30 19:13:21.339065 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.339030    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-token\" (UniqueName: \"kubernetes.io/projected/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-token\") pod \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") "
Oct 30 19:13:21.339122 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.339096    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istiod-ca-cert\") pod \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") "
Oct 30 19:13:21.339210 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.339191    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-credential-socket\") pod \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\" (UID: \"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4\") "
Oct 30 19:13:21.339437 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.339411    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-data" pod "b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" (UID: "b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:21.339683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.339656    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-workload-certs" pod "b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" (UID: "b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:21.339871 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.339841    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istiod-ca-cert" pod "b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" (UID: "b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:21.340071 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.340047    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-credential-socket" pod "b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" (UID: "b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:21.340245 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.340220    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:21.340341 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.340247    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:21.340341 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.340265    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:21.340341 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.340279    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:21.340341 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.340291    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:21.341975 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.341954    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-podinfo" pod "b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" (UID: "b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:13:21.345368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.345340    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-token" pod "b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" (UID: "b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:21.345465 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.345395    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-kube-api-access-xq7rb" pod "b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" (UID: "b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4"). InnerVolumeSpecName "kube-api-access-xq7rb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:21.345591 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.345569    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-envoy" pod "b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" (UID: "b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:21.441568 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.441532    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq7rb\" (UniqueName: \"kubernetes.io/projected/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-kube-api-access-xq7rb\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:21.441568 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.441563    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:21.441568 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.441574    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:21.441914 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:21.441583    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:22.250252 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:22.250217    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"
Oct 30 19:13:22.250809 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:22.250220    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn" event={"ID":"b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4","Type":"ContainerDied","Data":"7ffc1a9fe7dc8b907569685b7c84df802d9d692e123213806ec6f2cffca31a61"}
Oct 30 19:13:22.250809 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:22.250365    4544 scope.go:122] "RemoveContainer" containerID="14e7557314a12db33fe80d6446599c175210ba979d5778b68d022f52a3aab00b"
Oct 30 19:13:22.269129 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:13:22.269098    4544 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4ac89bb_06f0_4b1d_a5f2_98a3d59f37b4.slice/crio-7ffc1a9fe7dc8b907569685b7c84df802d9d692e123213806ec6f2cffca31a61\": RecentStats: unable to find data in memory cache]"
Oct 30 19:13:22.467433 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:22.467400    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"]
Oct 30 19:13:22.471737 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:22.471711    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5705f-openshift-default-64b647b647-wx4vn"]
Oct 30 19:13:22.641080 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:22.641048    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" path="/var/lib/kubelet/pods/b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4/volumes"
Oct 30 19:13:24.709662 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:24.709625    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz" podUID="a5dea2af-18e6-458e-9e34-561bb06ba3f6" containerName="payload-processing" containerID="cri-o://ff702f8226aefdf3d16a5d68782b5b84a4cf1137056c41947379ea0054211e9a" gracePeriod=25
Oct 30 19:13:24.736627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:24.735029    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"]
Oct 30 19:13:24.736627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:24.735323    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd" podUID="0686c324-b270-432b-9dbb-8d6ad419a957" containerName="istio-proxy" containerID="cri-o://26fb1c7b93a9e8105277b79fce53caf721944fe6e5fb87f9b8f59aab1a3aa7f5" gracePeriod=30
Oct 30 19:13:24.775492 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:24.775445    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"]
Oct 30 19:13:24.775767 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:24.775738    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7" podUID="edd86ca1-466c-4e45-8998-61d7e94fae7d" containerName="istio-proxy" containerID="cri-o://4a0e01c8d1b9c81a6fb01bcc321a4ce4c7628aa16eb44f09351a72fe5e4d6dce" gracePeriod=30
Oct 30 19:13:24.778347 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:24.778319    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"]
Oct 30 19:13:24.778562 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:24.778530    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9" podUID="e14048da-861d-4530-851e-8417b6cbf8bb" containerName="istio-proxy" containerID="cri-o://5a3f9d66bea7946fa68eb98db78807ce9666c883ad9fc152bd37e0e121245dfc" gracePeriod=30
Oct 30 19:13:24.815182 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:24.815145    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl" podUID="18c8ec61-e79e-4370-a890-477e9ba68f10" containerName="payload-pre-processing" containerID="cri-o://79e91c40363f48c56c65f16d0303b13bd64a1a186a639f5d18950d90093f99af" gracePeriod=25
Oct 30 19:13:24.825520 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:24.825490    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"]
Oct 30 19:13:24.825759 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:24.825736    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg" podUID="5ccae868-24e4-456f-ad9a-58c17d714b4e" containerName="istio-proxy" containerID="cri-o://bc52851088bdd7899f6d74ead1b274fb18331bea864a977142d27030b1410dbd" gracePeriod=30
Oct 30 19:13:25.285799 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.285765    4544 generic.go:356] "Generic (PLEG): container finished" podID="18c8ec61-e79e-4370-a890-477e9ba68f10" containerID="79e91c40363f48c56c65f16d0303b13bd64a1a186a639f5d18950d90093f99af" exitCode=0
Oct 30 19:13:25.286166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.286143    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl" event={"ID":"18c8ec61-e79e-4370-a890-477e9ba68f10","Type":"ContainerDied","Data":"79e91c40363f48c56c65f16d0303b13bd64a1a186a639f5d18950d90093f99af"}
Oct 30 19:13:25.289624 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.289578    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz"
Oct 30 19:13:25.290745 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.290684    4544 generic.go:356] "Generic (PLEG): container finished" podID="a5dea2af-18e6-458e-9e34-561bb06ba3f6" containerID="ff702f8226aefdf3d16a5d68782b5b84a4cf1137056c41947379ea0054211e9a" exitCode=0
Oct 30 19:13:25.290849 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.290781    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz" event={"ID":"a5dea2af-18e6-458e-9e34-561bb06ba3f6","Type":"ContainerDied","Data":"ff702f8226aefdf3d16a5d68782b5b84a4cf1137056c41947379ea0054211e9a"}
Oct 30 19:13:25.290849 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.290812    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz" event={"ID":"a5dea2af-18e6-458e-9e34-561bb06ba3f6","Type":"ContainerDied","Data":"4a5f13872f01c3f912f6642fe0673f61a8487156af3f07bc19d2485e708340b5"}
Oct 30 19:13:25.290849 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.290833    4544 scope.go:122] "RemoveContainer" containerID="ff702f8226aefdf3d16a5d68782b5b84a4cf1137056c41947379ea0054211e9a"
Oct 30 19:13:25.301904 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.301884    4544 scope.go:122] "RemoveContainer" containerID="ff702f8226aefdf3d16a5d68782b5b84a4cf1137056c41947379ea0054211e9a"
Oct 30 19:13:25.302197 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:13:25.302169    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff702f8226aefdf3d16a5d68782b5b84a4cf1137056c41947379ea0054211e9a\": container with ID starting with ff702f8226aefdf3d16a5d68782b5b84a4cf1137056c41947379ea0054211e9a not found: ID does not exist" containerID="ff702f8226aefdf3d16a5d68782b5b84a4cf1137056c41947379ea0054211e9a"
Oct 30 19:13:25.302287 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.302206    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff702f8226aefdf3d16a5d68782b5b84a4cf1137056c41947379ea0054211e9a"} err="failed to get container status \"ff702f8226aefdf3d16a5d68782b5b84a4cf1137056c41947379ea0054211e9a\": rpc error: code = NotFound desc = could not find container \"ff702f8226aefdf3d16a5d68782b5b84a4cf1137056c41947379ea0054211e9a\": container with ID starting with ff702f8226aefdf3d16a5d68782b5b84a4cf1137056c41947379ea0054211e9a not found: ID does not exist"
Oct 30 19:13:25.303727 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.303704    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl"
Oct 30 19:13:25.318757 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.318736    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5dea2af-18e6-458e-9e34-561bb06ba3f6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5dea2af-18e6-458e-9e34-561bb06ba3f6-plugins-config-volume\") pod \"a5dea2af-18e6-458e-9e34-561bb06ba3f6\" (UID: \"a5dea2af-18e6-458e-9e34-561bb06ba3f6\") "
Oct 30 19:13:25.318903 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.318890    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/18c8ec61-e79e-4370-a890-477e9ba68f10-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18c8ec61-e79e-4370-a890-477e9ba68f10-plugins-config-volume\") pod \"18c8ec61-e79e-4370-a890-477e9ba68f10\" (UID: \"18c8ec61-e79e-4370-a890-477e9ba68f10\") "
Oct 30 19:13:25.318968 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.318912    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18c8ec61-e79e-4370-a890-477e9ba68f10-kube-api-access-shp78\" (UniqueName: \"kubernetes.io/projected/18c8ec61-e79e-4370-a890-477e9ba68f10-kube-api-access-shp78\") pod \"18c8ec61-e79e-4370-a890-477e9ba68f10\" (UID: \"18c8ec61-e79e-4370-a890-477e9ba68f10\") "
Oct 30 19:13:25.319023 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.319002    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5dea2af-18e6-458e-9e34-561bb06ba3f6-kube-api-access-2vzdl\" (UniqueName: \"kubernetes.io/projected/a5dea2af-18e6-458e-9e34-561bb06ba3f6-kube-api-access-2vzdl\") pod \"a5dea2af-18e6-458e-9e34-561bb06ba3f6\" (UID: \"a5dea2af-18e6-458e-9e34-561bb06ba3f6\") "
Oct 30 19:13:25.319256 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.319232    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5dea2af-18e6-458e-9e34-561bb06ba3f6-plugins-config-volume" pod "a5dea2af-18e6-458e-9e34-561bb06ba3f6" (UID: "a5dea2af-18e6-458e-9e34-561bb06ba3f6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:25.319320 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.319268    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18c8ec61-e79e-4370-a890-477e9ba68f10-plugins-config-volume" pod "18c8ec61-e79e-4370-a890-477e9ba68f10" (UID: "18c8ec61-e79e-4370-a890-477e9ba68f10"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:25.319432 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.319417    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5dea2af-18e6-458e-9e34-561bb06ba3f6-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:25.319485 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.319435    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18c8ec61-e79e-4370-a890-477e9ba68f10-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:25.321082 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.321054    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18c8ec61-e79e-4370-a890-477e9ba68f10-kube-api-access-shp78" pod "18c8ec61-e79e-4370-a890-477e9ba68f10" (UID: "18c8ec61-e79e-4370-a890-477e9ba68f10"). InnerVolumeSpecName "kube-api-access-shp78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:25.321179 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.321140    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5dea2af-18e6-458e-9e34-561bb06ba3f6-kube-api-access-2vzdl" pod "a5dea2af-18e6-458e-9e34-561bb06ba3f6" (UID: "a5dea2af-18e6-458e-9e34-561bb06ba3f6"). InnerVolumeSpecName "kube-api-access-2vzdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:25.392640 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.392570    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"]
Oct 30 19:13:25.392900 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.392863    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6" podUID="0456a5cd-81b2-4dd1-9e36-a51808065ad0" containerName="istio-proxy" containerID="cri-o://5ec6d6e945b63f3120169945aa0a7bf106daea4bb0e5ee335c887551f0d04265" gracePeriod=30
Oct 30 19:13:25.420892 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.420859    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shp78\" (UniqueName: \"kubernetes.io/projected/18c8ec61-e79e-4370-a890-477e9ba68f10-kube-api-access-shp78\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:25.420892 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:25.420892    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vzdl\" (UniqueName: \"kubernetes.io/projected/a5dea2af-18e6-458e-9e34-561bb06ba3f6-kube-api-access-2vzdl\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:26.298582 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:26.298551    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl"
Oct 30 19:13:26.298582 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:26.298564    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl" event={"ID":"18c8ec61-e79e-4370-a890-477e9ba68f10","Type":"ContainerDied","Data":"51050f88a686c13f19fb4d5bab101d168cf809361424218dc51bf5e101e4822e"}
Oct 30 19:13:26.299230 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:26.298653    4544 scope.go:122] "RemoveContainer" containerID="79e91c40363f48c56c65f16d0303b13bd64a1a186a639f5d18950d90093f99af"
Oct 30 19:13:26.305524 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:26.305502    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz"
Oct 30 19:13:26.465651 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:26.465615    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl"]
Oct 30 19:13:26.470113 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:26.470058    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-2ef49a-77cd8b4c7f-cm5cl"]
Oct 30 19:13:26.585407 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:26.585322    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz"]
Oct 30 19:13:26.589073 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:26.589050    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2ef49a-7f85457dd-tqxbz"]
Oct 30 19:13:26.606590 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:26.606564    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="18c8ec61-e79e-4370-a890-477e9ba68f10" path="/var/lib/kubelet/pods/18c8ec61-e79e-4370-a890-477e9ba68f10/volumes"
Oct 30 19:13:26.607065 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:26.607049    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5dea2af-18e6-458e-9e34-561bb06ba3f6" path="/var/lib/kubelet/pods/a5dea2af-18e6-458e-9e34-561bb06ba3f6/volumes"
Oct 30 19:13:30.111855 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.111824    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:13:30.213416 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.213391    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:13:30.217183 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.217159    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0686c324-b270-432b-9dbb-8d6ad419a957-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0686c324-b270-432b-9dbb-8d6ad419a957-istiod-ca-cert\") pod \"0686c324-b270-432b-9dbb-8d6ad419a957\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") "
Oct 30 19:13:30.217333 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.217283    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0686c324-b270-432b-9dbb-8d6ad419a957-kube-api-access-2v6h6\" (UniqueName: \"kubernetes.io/projected/0686c324-b270-432b-9dbb-8d6ad419a957-kube-api-access-2v6h6\") pod \"0686c324-b270-432b-9dbb-8d6ad419a957\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") "
Oct 30 19:13:30.217403 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.217347    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0686c324-b270-432b-9dbb-8d6ad419a957-istio-token\" (UniqueName: \"kubernetes.io/projected/0686c324-b270-432b-9dbb-8d6ad419a957-istio-token\") pod \"0686c324-b270-432b-9dbb-8d6ad419a957\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") "
Oct 30 19:13:30.217403 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.217366    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-istio-data\") pod \"0686c324-b270-432b-9dbb-8d6ad419a957\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") "
Oct 30 19:13:30.217403 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.217386    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-workload-socket\") pod \"0686c324-b270-432b-9dbb-8d6ad419a957\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") "
Oct 30 19:13:30.217787 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.217405    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-credential-socket\") pod \"0686c324-b270-432b-9dbb-8d6ad419a957\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") "
Oct 30 19:13:30.217787 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.217427    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0686c324-b270-432b-9dbb-8d6ad419a957-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0686c324-b270-432b-9dbb-8d6ad419a957-istio-podinfo\") pod \"0686c324-b270-432b-9dbb-8d6ad419a957\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") "
Oct 30 19:13:30.217787 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.217507    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-workload-certs\") pod \"0686c324-b270-432b-9dbb-8d6ad419a957\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") "
Oct 30 19:13:30.217787 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.217547    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-istio-envoy\") pod \"0686c324-b270-432b-9dbb-8d6ad419a957\" (UID: \"0686c324-b270-432b-9dbb-8d6ad419a957\") "
Oct 30 19:13:30.218420 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.218381    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-credential-socket" pod "0686c324-b270-432b-9dbb-8d6ad419a957" (UID: "0686c324-b270-432b-9dbb-8d6ad419a957"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.218704 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.218668    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-workload-socket" pod "0686c324-b270-432b-9dbb-8d6ad419a957" (UID: "0686c324-b270-432b-9dbb-8d6ad419a957"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.219103 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.219079    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0686c324-b270-432b-9dbb-8d6ad419a957-istiod-ca-cert" pod "0686c324-b270-432b-9dbb-8d6ad419a957" (UID: "0686c324-b270-432b-9dbb-8d6ad419a957"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:30.219198 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.219142    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-workload-certs" pod "0686c324-b270-432b-9dbb-8d6ad419a957" (UID: "0686c324-b270-432b-9dbb-8d6ad419a957"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.219405 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.219382    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-istio-data" pod "0686c324-b270-432b-9dbb-8d6ad419a957" (UID: "0686c324-b270-432b-9dbb-8d6ad419a957"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.221141 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.221063    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0686c324-b270-432b-9dbb-8d6ad419a957-kube-api-access-2v6h6" pod "0686c324-b270-432b-9dbb-8d6ad419a957" (UID: "0686c324-b270-432b-9dbb-8d6ad419a957"). InnerVolumeSpecName "kube-api-access-2v6h6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:30.221141 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.221101    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0686c324-b270-432b-9dbb-8d6ad419a957-istio-token" pod "0686c324-b270-432b-9dbb-8d6ad419a957" (UID: "0686c324-b270-432b-9dbb-8d6ad419a957"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:30.221289 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.221247    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0686c324-b270-432b-9dbb-8d6ad419a957-istio-podinfo" pod "0686c324-b270-432b-9dbb-8d6ad419a957" (UID: "0686c324-b270-432b-9dbb-8d6ad419a957"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:13:30.222968 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.222828    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-istio-envoy" pod "0686c324-b270-432b-9dbb-8d6ad419a957" (UID: "0686c324-b270-432b-9dbb-8d6ad419a957"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.228219 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.228197    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:13:30.230408 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.230390    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:13:30.319188 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319106    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-istio-envoy\") pod \"e14048da-861d-4530-851e-8417b6cbf8bb\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") "
Oct 30 19:13:30.319188 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319142    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-workload-certs\") pod \"edd86ca1-466c-4e45-8998-61d7e94fae7d\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") "
Oct 30 19:13:30.319188 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319164    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-token\" (UniqueName: \"kubernetes.io/projected/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-token\") pod \"5ccae868-24e4-456f-ad9a-58c17d714b4e\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") "
Oct 30 19:13:30.319479 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319197    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-workload-certs\") pod \"e14048da-861d-4530-851e-8417b6cbf8bb\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") "
Oct 30 19:13:30.319479 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319220    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e14048da-861d-4530-851e-8417b6cbf8bb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e14048da-861d-4530-851e-8417b6cbf8bb-istio-podinfo\") pod \"e14048da-861d-4530-851e-8417b6cbf8bb\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") "
Oct 30 19:13:30.319479 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319257    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-workload-socket\") pod \"5ccae868-24e4-456f-ad9a-58c17d714b4e\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") "
Oct 30 19:13:30.319479 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319296    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-data\") pod \"edd86ca1-466c-4e45-8998-61d7e94fae7d\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") "
Oct 30 19:13:30.319479 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319362    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-workload-socket\") pod \"e14048da-861d-4530-851e-8417b6cbf8bb\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") "
Oct 30 19:13:30.319479 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319409    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edd86ca1-466c-4e45-8998-61d7e94fae7d-kube-api-access-9wnzd\" (UniqueName: \"kubernetes.io/projected/edd86ca1-466c-4e45-8998-61d7e94fae7d-kube-api-access-9wnzd\") pod \"edd86ca1-466c-4e45-8998-61d7e94fae7d\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") "
Oct 30 19:13:30.319479 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319468    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ccae868-24e4-456f-ad9a-58c17d714b4e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ccae868-24e4-456f-ad9a-58c17d714b4e-istiod-ca-cert\") pod \"5ccae868-24e4-456f-ad9a-58c17d714b4e\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") "
Oct 30 19:13:30.319880 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319494    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e14048da-861d-4530-851e-8417b6cbf8bb-istio-token\" (UniqueName: \"kubernetes.io/projected/e14048da-861d-4530-851e-8417b6cbf8bb-istio-token\") pod \"e14048da-861d-4530-851e-8417b6cbf8bb\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") "
Oct 30 19:13:30.319880 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319497    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-workload-certs" pod "e14048da-861d-4530-851e-8417b6cbf8bb" (UID: "e14048da-861d-4530-851e-8417b6cbf8bb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.319880 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319503    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-workload-certs" pod "edd86ca1-466c-4e45-8998-61d7e94fae7d" (UID: "edd86ca1-466c-4e45-8998-61d7e94fae7d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.319880 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319535    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-token\" (UniqueName: \"kubernetes.io/projected/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-token\") pod \"edd86ca1-466c-4e45-8998-61d7e94fae7d\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") "
Oct 30 19:13:30.319880 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319578    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-credential-socket\") pod \"edd86ca1-466c-4e45-8998-61d7e94fae7d\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") "
Oct 30 19:13:30.319880 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319641    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-podinfo\") pod \"5ccae868-24e4-456f-ad9a-58c17d714b4e\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") "
Oct 30 19:13:30.319880 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319670    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e14048da-861d-4530-851e-8417b6cbf8bb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e14048da-861d-4530-851e-8417b6cbf8bb-istiod-ca-cert\") pod \"e14048da-861d-4530-851e-8417b6cbf8bb\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") "
Oct 30 19:13:30.319880 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319704    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ccae868-24e4-456f-ad9a-58c17d714b4e-kube-api-access-4bjdd\" (UniqueName: \"kubernetes.io/projected/5ccae868-24e4-456f-ad9a-58c17d714b4e-kube-api-access-4bjdd\") pod \"5ccae868-24e4-456f-ad9a-58c17d714b4e\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") "
Oct 30 19:13:30.319880 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319822    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/edd86ca1-466c-4e45-8998-61d7e94fae7d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edd86ca1-466c-4e45-8998-61d7e94fae7d-istiod-ca-cert\") pod \"edd86ca1-466c-4e45-8998-61d7e94fae7d\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") "
Oct 30 19:13:30.320352 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319903    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e14048da-861d-4530-851e-8417b6cbf8bb-kube-api-access-xjr4q\" (UniqueName: \"kubernetes.io/projected/e14048da-861d-4530-851e-8417b6cbf8bb-kube-api-access-xjr4q\") pod \"e14048da-861d-4530-851e-8417b6cbf8bb\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") "
Oct 30 19:13:30.320352 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319967    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-envoy\") pod \"5ccae868-24e4-456f-ad9a-58c17d714b4e\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") "
Oct 30 19:13:30.320352 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.320020    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-envoy\") pod \"edd86ca1-466c-4e45-8998-61d7e94fae7d\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") "
Oct 30 19:13:30.320352 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.320052    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-credential-socket\") pod \"5ccae868-24e4-456f-ad9a-58c17d714b4e\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") "
Oct 30 19:13:30.320352 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.320079    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-workload-certs\") pod \"5ccae868-24e4-456f-ad9a-58c17d714b4e\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") "
Oct 30 19:13:30.320352 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.320134    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-data\") pod \"5ccae868-24e4-456f-ad9a-58c17d714b4e\" (UID: \"5ccae868-24e4-456f-ad9a-58c17d714b4e\") "
Oct 30 19:13:30.320352 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.320163    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-istio-data\") pod \"e14048da-861d-4530-851e-8417b6cbf8bb\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") "
Oct 30 19:13:30.320352 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.320195    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-workload-socket\") pod \"edd86ca1-466c-4e45-8998-61d7e94fae7d\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") "
Oct 30 19:13:30.320352 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.320240    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-podinfo\") pod \"edd86ca1-466c-4e45-8998-61d7e94fae7d\" (UID: \"edd86ca1-466c-4e45-8998-61d7e94fae7d\") "
Oct 30 19:13:30.320352 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.320280    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-credential-socket\") pod \"e14048da-861d-4530-851e-8417b6cbf8bb\" (UID: \"e14048da-861d-4530-851e-8417b6cbf8bb\") "
Oct 30 19:13:30.321009 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.320984    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.321083 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.321017    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.321083 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.321033    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0686c324-b270-432b-9dbb-8d6ad419a957-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.321083 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.321046    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.321083 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.321061    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.321083 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.321076    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2v6h6\" (UniqueName: \"kubernetes.io/projected/0686c324-b270-432b-9dbb-8d6ad419a957-kube-api-access-2v6h6\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.321353 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.321092    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0686c324-b270-432b-9dbb-8d6ad419a957-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.321353 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.321107    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.321353 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.321121    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.321353 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.321134    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0686c324-b270-432b-9dbb-8d6ad419a957-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.321353 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.321147    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0686c324-b270-432b-9dbb-8d6ad419a957-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.322578 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319699    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-workload-socket" pod "e14048da-861d-4530-851e-8417b6cbf8bb" (UID: "e14048da-861d-4530-851e-8417b6cbf8bb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.322727 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319720    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-data" pod "edd86ca1-466c-4e45-8998-61d7e94fae7d" (UID: "edd86ca1-466c-4e45-8998-61d7e94fae7d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.322727 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319804    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-workload-socket" pod "5ccae868-24e4-456f-ad9a-58c17d714b4e" (UID: "5ccae868-24e4-456f-ad9a-58c17d714b4e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.322727 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.319935    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ccae868-24e4-456f-ad9a-58c17d714b4e-istiod-ca-cert" pod "5ccae868-24e4-456f-ad9a-58c17d714b4e" (UID: "5ccae868-24e4-456f-ad9a-58c17d714b4e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:30.322727 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.320235    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-credential-socket" pod "edd86ca1-466c-4e45-8998-61d7e94fae7d" (UID: "edd86ca1-466c-4e45-8998-61d7e94fae7d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.322727 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.321368    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-credential-socket" pod "e14048da-861d-4530-851e-8417b6cbf8bb" (UID: "e14048da-861d-4530-851e-8417b6cbf8bb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.322727 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.322547    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edd86ca1-466c-4e45-8998-61d7e94fae7d-istiod-ca-cert" pod "edd86ca1-466c-4e45-8998-61d7e94fae7d" (UID: "edd86ca1-466c-4e45-8998-61d7e94fae7d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:30.323065 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.322972    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-token" pod "edd86ca1-466c-4e45-8998-61d7e94fae7d" (UID: "edd86ca1-466c-4e45-8998-61d7e94fae7d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:30.323456 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.323179    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-data" pod "5ccae868-24e4-456f-ad9a-58c17d714b4e" (UID: "5ccae868-24e4-456f-ad9a-58c17d714b4e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.323456 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.323355    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-credential-socket" pod "5ccae868-24e4-456f-ad9a-58c17d714b4e" (UID: "5ccae868-24e4-456f-ad9a-58c17d714b4e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.323643 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.323510    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-token" pod "5ccae868-24e4-456f-ad9a-58c17d714b4e" (UID: "5ccae868-24e4-456f-ad9a-58c17d714b4e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:30.323643 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.323631    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-istio-envoy" pod "e14048da-861d-4530-851e-8417b6cbf8bb" (UID: "e14048da-861d-4530-851e-8417b6cbf8bb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.323775 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.323649    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-workload-certs" pod "5ccae868-24e4-456f-ad9a-58c17d714b4e" (UID: "5ccae868-24e4-456f-ad9a-58c17d714b4e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.323775 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.323695    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e14048da-861d-4530-851e-8417b6cbf8bb-istio-token" pod "e14048da-861d-4530-851e-8417b6cbf8bb" (UID: "e14048da-861d-4530-851e-8417b6cbf8bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:30.323905 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.323846    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-workload-socket" pod "edd86ca1-466c-4e45-8998-61d7e94fae7d" (UID: "edd86ca1-466c-4e45-8998-61d7e94fae7d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.324304 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.324060    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-istio-data" pod "e14048da-861d-4530-851e-8417b6cbf8bb" (UID: "e14048da-861d-4530-851e-8417b6cbf8bb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.324407 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.324349    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edd86ca1-466c-4e45-8998-61d7e94fae7d-kube-api-access-9wnzd" pod "edd86ca1-466c-4e45-8998-61d7e94fae7d" (UID: "edd86ca1-466c-4e45-8998-61d7e94fae7d"). InnerVolumeSpecName "kube-api-access-9wnzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:30.324482 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.324414    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-envoy" pod "edd86ca1-466c-4e45-8998-61d7e94fae7d" (UID: "edd86ca1-466c-4e45-8998-61d7e94fae7d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.324918 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.324658    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ccae868-24e4-456f-ad9a-58c17d714b4e-kube-api-access-4bjdd" pod "5ccae868-24e4-456f-ad9a-58c17d714b4e" (UID: "5ccae868-24e4-456f-ad9a-58c17d714b4e"). InnerVolumeSpecName "kube-api-access-4bjdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:30.324918 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.324674    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e14048da-861d-4530-851e-8417b6cbf8bb-istiod-ca-cert" pod "e14048da-861d-4530-851e-8417b6cbf8bb" (UID: "e14048da-861d-4530-851e-8417b6cbf8bb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:30.325245 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.325201    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e14048da-861d-4530-851e-8417b6cbf8bb-istio-podinfo" pod "e14048da-861d-4530-851e-8417b6cbf8bb" (UID: "e14048da-861d-4530-851e-8417b6cbf8bb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:13:30.325345 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.325288    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-envoy" pod "5ccae868-24e4-456f-ad9a-58c17d714b4e" (UID: "5ccae868-24e4-456f-ad9a-58c17d714b4e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.325434 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.325409    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-podinfo" pod "5ccae868-24e4-456f-ad9a-58c17d714b4e" (UID: "5ccae868-24e4-456f-ad9a-58c17d714b4e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:13:30.326218 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.326194    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-podinfo" pod "edd86ca1-466c-4e45-8998-61d7e94fae7d" (UID: "edd86ca1-466c-4e45-8998-61d7e94fae7d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:13:30.326284 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.326258    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e14048da-861d-4530-851e-8417b6cbf8bb-kube-api-access-xjr4q" pod "e14048da-861d-4530-851e-8417b6cbf8bb" (UID: "e14048da-861d-4530-851e-8417b6cbf8bb"). InnerVolumeSpecName "kube-api-access-xjr4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:30.341039 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.341012    4544 generic.go:356] "Generic (PLEG): container finished" podID="0686c324-b270-432b-9dbb-8d6ad419a957" containerID="26fb1c7b93a9e8105277b79fce53caf721944fe6e5fb87f9b8f59aab1a3aa7f5" exitCode=0
Oct 30 19:13:30.341155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.341064    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"
Oct 30 19:13:30.341155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.341102    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd" event={"ID":"0686c324-b270-432b-9dbb-8d6ad419a957","Type":"ContainerDied","Data":"26fb1c7b93a9e8105277b79fce53caf721944fe6e5fb87f9b8f59aab1a3aa7f5"}
Oct 30 19:13:30.341255 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.341163    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd" event={"ID":"0686c324-b270-432b-9dbb-8d6ad419a957","Type":"ContainerDied","Data":"05a2873b0c04aee33f3d618a972aff59d32b8ca113cbb4761eb33b288c1ef0e4"}
Oct 30 19:13:30.341255 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.341181    4544 scope.go:122] "RemoveContainer" containerID="26fb1c7b93a9e8105277b79fce53caf721944fe6e5fb87f9b8f59aab1a3aa7f5"
Oct 30 19:13:30.342666 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.342641    4544 generic.go:356] "Generic (PLEG): container finished" podID="edd86ca1-466c-4e45-8998-61d7e94fae7d" containerID="4a0e01c8d1b9c81a6fb01bcc321a4ce4c7628aa16eb44f09351a72fe5e4d6dce" exitCode=0
Oct 30 19:13:30.342807 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.342793    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"
Oct 30 19:13:30.342871 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.342829    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7" event={"ID":"edd86ca1-466c-4e45-8998-61d7e94fae7d","Type":"ContainerDied","Data":"4a0e01c8d1b9c81a6fb01bcc321a4ce4c7628aa16eb44f09351a72fe5e4d6dce"}
Oct 30 19:13:30.342929 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.342879    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7" event={"ID":"edd86ca1-466c-4e45-8998-61d7e94fae7d","Type":"ContainerDied","Data":"a98f5d72b4c14a9805084bbccf48e1a5f847887e8c1fb2dd89c4768eee7c7145"}
Oct 30 19:13:30.344146 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.344123    4544 generic.go:356] "Generic (PLEG): container finished" podID="e14048da-861d-4530-851e-8417b6cbf8bb" containerID="5a3f9d66bea7946fa68eb98db78807ce9666c883ad9fc152bd37e0e121245dfc" exitCode=0
Oct 30 19:13:30.344288 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.344272    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9" event={"ID":"e14048da-861d-4530-851e-8417b6cbf8bb","Type":"ContainerDied","Data":"5a3f9d66bea7946fa68eb98db78807ce9666c883ad9fc152bd37e0e121245dfc"}
Oct 30 19:13:30.344348 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.344313    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9" event={"ID":"e14048da-861d-4530-851e-8417b6cbf8bb","Type":"ContainerDied","Data":"47e964ac8aa0d40fd3f1e27ea5d5e680f9814d7e71a6055995242fa374b60f83"}
Oct 30 19:13:30.344412 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.344374    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"
Oct 30 19:13:30.348473 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.348449    4544 generic.go:356] "Generic (PLEG): container finished" podID="5ccae868-24e4-456f-ad9a-58c17d714b4e" containerID="bc52851088bdd7899f6d74ead1b274fb18331bea864a977142d27030b1410dbd" exitCode=0
Oct 30 19:13:30.348553 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.348522    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg" event={"ID":"5ccae868-24e4-456f-ad9a-58c17d714b4e","Type":"ContainerDied","Data":"bc52851088bdd7899f6d74ead1b274fb18331bea864a977142d27030b1410dbd"}
Oct 30 19:13:30.348591 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.348557    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg" event={"ID":"5ccae868-24e4-456f-ad9a-58c17d714b4e","Type":"ContainerDied","Data":"78f1d2d34ea34308b5863acdf93d3813442c2c3bf3b17807ec937600eaed115e"}
Oct 30 19:13:30.348658 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.348634    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"
Oct 30 19:13:30.353258 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.353239    4544 scope.go:122] "RemoveContainer" containerID="26fb1c7b93a9e8105277b79fce53caf721944fe6e5fb87f9b8f59aab1a3aa7f5"
Oct 30 19:13:30.354745 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:13:30.354717    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26fb1c7b93a9e8105277b79fce53caf721944fe6e5fb87f9b8f59aab1a3aa7f5\": container with ID starting with 26fb1c7b93a9e8105277b79fce53caf721944fe6e5fb87f9b8f59aab1a3aa7f5 not found: ID does not exist" containerID="26fb1c7b93a9e8105277b79fce53caf721944fe6e5fb87f9b8f59aab1a3aa7f5"
Oct 30 19:13:30.354844 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.354754    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26fb1c7b93a9e8105277b79fce53caf721944fe6e5fb87f9b8f59aab1a3aa7f5"} err="failed to get container status \"26fb1c7b93a9e8105277b79fce53caf721944fe6e5fb87f9b8f59aab1a3aa7f5\": rpc error: code = NotFound desc = could not find container \"26fb1c7b93a9e8105277b79fce53caf721944fe6e5fb87f9b8f59aab1a3aa7f5\": container with ID starting with 26fb1c7b93a9e8105277b79fce53caf721944fe6e5fb87f9b8f59aab1a3aa7f5 not found: ID does not exist"
Oct 30 19:13:30.354844 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.354780    4544 scope.go:122] "RemoveContainer" containerID="4a0e01c8d1b9c81a6fb01bcc321a4ce4c7628aa16eb44f09351a72fe5e4d6dce"
Oct 30 19:13:30.367638 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.367622    4544 scope.go:122] "RemoveContainer" containerID="4a0e01c8d1b9c81a6fb01bcc321a4ce4c7628aa16eb44f09351a72fe5e4d6dce"
Oct 30 19:13:30.367945 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:13:30.367903    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a0e01c8d1b9c81a6fb01bcc321a4ce4c7628aa16eb44f09351a72fe5e4d6dce\": container with ID starting with 4a0e01c8d1b9c81a6fb01bcc321a4ce4c7628aa16eb44f09351a72fe5e4d6dce not found: ID does not exist" containerID="4a0e01c8d1b9c81a6fb01bcc321a4ce4c7628aa16eb44f09351a72fe5e4d6dce"
Oct 30 19:13:30.368083 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.368055    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a0e01c8d1b9c81a6fb01bcc321a4ce4c7628aa16eb44f09351a72fe5e4d6dce"} err="failed to get container status \"4a0e01c8d1b9c81a6fb01bcc321a4ce4c7628aa16eb44f09351a72fe5e4d6dce\": rpc error: code = NotFound desc = could not find container \"4a0e01c8d1b9c81a6fb01bcc321a4ce4c7628aa16eb44f09351a72fe5e4d6dce\": container with ID starting with 4a0e01c8d1b9c81a6fb01bcc321a4ce4c7628aa16eb44f09351a72fe5e4d6dce not found: ID does not exist"
Oct 30 19:13:30.368163 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.368086    4544 scope.go:122] "RemoveContainer" containerID="5a3f9d66bea7946fa68eb98db78807ce9666c883ad9fc152bd37e0e121245dfc"
Oct 30 19:13:30.378319 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.378302    4544 scope.go:122] "RemoveContainer" containerID="5a3f9d66bea7946fa68eb98db78807ce9666c883ad9fc152bd37e0e121245dfc"
Oct 30 19:13:30.378632 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:13:30.378589    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a3f9d66bea7946fa68eb98db78807ce9666c883ad9fc152bd37e0e121245dfc\": container with ID starting with 5a3f9d66bea7946fa68eb98db78807ce9666c883ad9fc152bd37e0e121245dfc not found: ID does not exist" containerID="5a3f9d66bea7946fa68eb98db78807ce9666c883ad9fc152bd37e0e121245dfc"
Oct 30 19:13:30.378740 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.378635    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a3f9d66bea7946fa68eb98db78807ce9666c883ad9fc152bd37e0e121245dfc"} err="failed to get container status \"5a3f9d66bea7946fa68eb98db78807ce9666c883ad9fc152bd37e0e121245dfc\": rpc error: code = NotFound desc = could not find container \"5a3f9d66bea7946fa68eb98db78807ce9666c883ad9fc152bd37e0e121245dfc\": container with ID starting with 5a3f9d66bea7946fa68eb98db78807ce9666c883ad9fc152bd37e0e121245dfc not found: ID does not exist"
Oct 30 19:13:30.378740 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.378651    4544 scope.go:122] "RemoveContainer" containerID="bc52851088bdd7899f6d74ead1b274fb18331bea864a977142d27030b1410dbd"
Oct 30 19:13:30.387014 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.386983    4544 scope.go:122] "RemoveContainer" containerID="bc52851088bdd7899f6d74ead1b274fb18331bea864a977142d27030b1410dbd"
Oct 30 19:13:30.387274 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:13:30.387254    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc52851088bdd7899f6d74ead1b274fb18331bea864a977142d27030b1410dbd\": container with ID starting with bc52851088bdd7899f6d74ead1b274fb18331bea864a977142d27030b1410dbd not found: ID does not exist" containerID="bc52851088bdd7899f6d74ead1b274fb18331bea864a977142d27030b1410dbd"
Oct 30 19:13:30.387367 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.387277    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc52851088bdd7899f6d74ead1b274fb18331bea864a977142d27030b1410dbd"} err="failed to get container status \"bc52851088bdd7899f6d74ead1b274fb18331bea864a977142d27030b1410dbd\": rpc error: code = NotFound desc = could not find container \"bc52851088bdd7899f6d74ead1b274fb18331bea864a977142d27030b1410dbd\": container with ID starting with bc52851088bdd7899f6d74ead1b274fb18331bea864a977142d27030b1410dbd not found: ID does not exist"
Oct 30 19:13:30.422434 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422404    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422542 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422438    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e14048da-861d-4530-851e-8417b6cbf8bb-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422542 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422457    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bjdd\" (UniqueName: \"kubernetes.io/projected/5ccae868-24e4-456f-ad9a-58c17d714b4e-kube-api-access-4bjdd\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422542 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422473    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edd86ca1-466c-4e45-8998-61d7e94fae7d-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422542 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422487    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjr4q\" (UniqueName: \"kubernetes.io/projected/e14048da-861d-4530-851e-8417b6cbf8bb-kube-api-access-xjr4q\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422542 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422501    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422542 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422513    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422542 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422527    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422542 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422540    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422553    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422567    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422580    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422592    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422642    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422657    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422671    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ccae868-24e4-456f-ad9a-58c17d714b4e-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422684    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e14048da-861d-4530-851e-8417b6cbf8bb-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422698    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ccae868-24e4-456f-ad9a-58c17d714b4e-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422711    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422724    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e14048da-861d-4530-851e-8417b6cbf8bb-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422738    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wnzd\" (UniqueName: \"kubernetes.io/projected/edd86ca1-466c-4e45-8998-61d7e94fae7d-kube-api-access-9wnzd\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422751    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ccae868-24e4-456f-ad9a-58c17d714b4e-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422765    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e14048da-861d-4530-851e-8417b6cbf8bb-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422778    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edd86ca1-466c-4e45-8998-61d7e94fae7d-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.422819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.422792    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edd86ca1-466c-4e45-8998-61d7e94fae7d-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.667420 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.667386    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:13:30.726151 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.726123    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-workload-socket\") pod \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") "
Oct 30 19:13:30.726327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.726191    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-envoy\") pod \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") "
Oct 30 19:13:30.726327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.726215    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-data\") pod \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") "
Oct 30 19:13:30.726327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.726273    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-podinfo\") pod \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") "
Oct 30 19:13:30.726327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.726297    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istiod-ca-cert\") pod \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") "
Oct 30 19:13:30.726327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.726323    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-credential-socket\") pod \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") "
Oct 30 19:13:30.726597 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.726371    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0456a5cd-81b2-4dd1-9e36-a51808065ad0-kube-api-access-qlpdt\" (UniqueName: \"kubernetes.io/projected/0456a5cd-81b2-4dd1-9e36-a51808065ad0-kube-api-access-qlpdt\") pod \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") "
Oct 30 19:13:30.726597 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.726379    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-workload-socket" pod "0456a5cd-81b2-4dd1-9e36-a51808065ad0" (UID: "0456a5cd-81b2-4dd1-9e36-a51808065ad0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.726597 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.726434    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-workload-certs\") pod \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") "
Oct 30 19:13:30.726597 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.726498    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-token\" (UniqueName: \"kubernetes.io/projected/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-token\") pod \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\" (UID: \"0456a5cd-81b2-4dd1-9e36-a51808065ad0\") "
Oct 30 19:13:30.726597 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.726547    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-data" pod "0456a5cd-81b2-4dd1-9e36-a51808065ad0" (UID: "0456a5cd-81b2-4dd1-9e36-a51808065ad0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.726879 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.726671    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-credential-socket" pod "0456a5cd-81b2-4dd1-9e36-a51808065ad0" (UID: "0456a5cd-81b2-4dd1-9e36-a51808065ad0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.726879 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.726770    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istiod-ca-cert" pod "0456a5cd-81b2-4dd1-9e36-a51808065ad0" (UID: "0456a5cd-81b2-4dd1-9e36-a51808065ad0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:30.726987 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.726938    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-workload-certs" pod "0456a5cd-81b2-4dd1-9e36-a51808065ad0" (UID: "0456a5cd-81b2-4dd1-9e36-a51808065ad0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.727331 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.727307    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.727428 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.727373    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.727428 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.727393    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.727428 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.727405    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.727428 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.727424    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.728778 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.728751    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0456a5cd-81b2-4dd1-9e36-a51808065ad0-kube-api-access-qlpdt" pod "0456a5cd-81b2-4dd1-9e36-a51808065ad0" (UID: "0456a5cd-81b2-4dd1-9e36-a51808065ad0"). InnerVolumeSpecName "kube-api-access-qlpdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:30.728883 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.728773    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-envoy" pod "0456a5cd-81b2-4dd1-9e36-a51808065ad0" (UID: "0456a5cd-81b2-4dd1-9e36-a51808065ad0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:30.728953 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.728893    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-podinfo" pod "0456a5cd-81b2-4dd1-9e36-a51808065ad0" (UID: "0456a5cd-81b2-4dd1-9e36-a51808065ad0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:13:30.729032 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.729013    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-token" pod "0456a5cd-81b2-4dd1-9e36-a51808065ad0" (UID: "0456a5cd-81b2-4dd1-9e36-a51808065ad0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:30.759939 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.759905    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"]
Oct 30 19:13:30.764576 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.764550    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-j76r7"]
Oct 30 19:13:30.828322 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.828296    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.828322 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.828323    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.828473 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.828338    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlpdt\" (UniqueName: \"kubernetes.io/projected/0456a5cd-81b2-4dd1-9e36-a51808065ad0-kube-api-access-qlpdt\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.828473 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.828351    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0456a5cd-81b2-4dd1-9e36-a51808065ad0-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:30.875268 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.875199    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"]
Oct 30 19:13:30.881385 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.881361    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-zp4pg"]
Oct 30 19:13:30.994690 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.994660    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"]
Oct 30 19:13:30.999004 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:30.998972    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-2jstd"]
Oct 30 19:13:31.107855 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:31.107820    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"]
Oct 30 19:13:31.112621 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:31.112580    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-f22z9"]
Oct 30 19:13:31.360055 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:31.360019    4544 generic.go:356] "Generic (PLEG): container finished" podID="0456a5cd-81b2-4dd1-9e36-a51808065ad0" containerID="5ec6d6e945b63f3120169945aa0a7bf106daea4bb0e5ee335c887551f0d04265" exitCode=0
Oct 30 19:13:31.360267 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:31.360068    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6" event={"ID":"0456a5cd-81b2-4dd1-9e36-a51808065ad0","Type":"ContainerDied","Data":"5ec6d6e945b63f3120169945aa0a7bf106daea4bb0e5ee335c887551f0d04265"}
Oct 30 19:13:31.360267 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:31.360091    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"
Oct 30 19:13:31.360267 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:31.360135    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6" event={"ID":"0456a5cd-81b2-4dd1-9e36-a51808065ad0","Type":"ContainerDied","Data":"2f0a733ee02c48ef0152db5a90e164d91a91e19557c8d2e9441f27484cadbc26"}
Oct 30 19:13:31.360267 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:31.360160    4544 scope.go:122] "RemoveContainer" containerID="5ec6d6e945b63f3120169945aa0a7bf106daea4bb0e5ee335c887551f0d04265"
Oct 30 19:13:31.375660 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:31.375637    4544 scope.go:122] "RemoveContainer" containerID="2fd67f2c017f50ab596a11be49a591b307b0a278e1c0a7e63434ee36a8e8b255"
Oct 30 19:13:31.390332 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:31.390311    4544 scope.go:122] "RemoveContainer" containerID="5ec6d6e945b63f3120169945aa0a7bf106daea4bb0e5ee335c887551f0d04265"
Oct 30 19:13:31.390636 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:13:31.390612    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ec6d6e945b63f3120169945aa0a7bf106daea4bb0e5ee335c887551f0d04265\": container with ID starting with 5ec6d6e945b63f3120169945aa0a7bf106daea4bb0e5ee335c887551f0d04265 not found: ID does not exist" containerID="5ec6d6e945b63f3120169945aa0a7bf106daea4bb0e5ee335c887551f0d04265"
Oct 30 19:13:31.390719 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:31.390643    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ec6d6e945b63f3120169945aa0a7bf106daea4bb0e5ee335c887551f0d04265"} err="failed to get container status \"5ec6d6e945b63f3120169945aa0a7bf106daea4bb0e5ee335c887551f0d04265\": rpc error: code = NotFound desc = could not find container \"5ec6d6e945b63f3120169945aa0a7bf106daea4bb0e5ee335c887551f0d04265\": container with ID starting with 5ec6d6e945b63f3120169945aa0a7bf106daea4bb0e5ee335c887551f0d04265 not found: ID does not exist"
Oct 30 19:13:31.390719 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:31.390662    4544 scope.go:122] "RemoveContainer" containerID="2fd67f2c017f50ab596a11be49a591b307b0a278e1c0a7e63434ee36a8e8b255"
Oct 30 19:13:31.390973 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:13:31.390952    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fd67f2c017f50ab596a11be49a591b307b0a278e1c0a7e63434ee36a8e8b255\": container with ID starting with 2fd67f2c017f50ab596a11be49a591b307b0a278e1c0a7e63434ee36a8e8b255 not found: ID does not exist" containerID="2fd67f2c017f50ab596a11be49a591b307b0a278e1c0a7e63434ee36a8e8b255"
Oct 30 19:13:31.391062 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:31.390979    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fd67f2c017f50ab596a11be49a591b307b0a278e1c0a7e63434ee36a8e8b255"} err="failed to get container status \"2fd67f2c017f50ab596a11be49a591b307b0a278e1c0a7e63434ee36a8e8b255\": rpc error: code = NotFound desc = could not find container \"2fd67f2c017f50ab596a11be49a591b307b0a278e1c0a7e63434ee36a8e8b255\": container with ID starting with 2fd67f2c017f50ab596a11be49a591b307b0a278e1c0a7e63434ee36a8e8b255 not found: ID does not exist"
Oct 30 19:13:31.531891 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:31.531840    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"]
Oct 30 19:13:31.551907 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:31.551874    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-071cf0-openshift-default-56c9db54d5-xqvm6"]
Oct 30 19:13:32.603693 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:32.603658    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0456a5cd-81b2-4dd1-9e36-a51808065ad0" path="/var/lib/kubelet/pods/0456a5cd-81b2-4dd1-9e36-a51808065ad0/volumes"
Oct 30 19:13:32.604308 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:32.604132    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0686c324-b270-432b-9dbb-8d6ad419a957" path="/var/lib/kubelet/pods/0686c324-b270-432b-9dbb-8d6ad419a957/volumes"
Oct 30 19:13:32.604540 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:32.604523    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ccae868-24e4-456f-ad9a-58c17d714b4e" path="/var/lib/kubelet/pods/5ccae868-24e4-456f-ad9a-58c17d714b4e/volumes"
Oct 30 19:13:32.605301 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:32.605279    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e14048da-861d-4530-851e-8417b6cbf8bb" path="/var/lib/kubelet/pods/e14048da-861d-4530-851e-8417b6cbf8bb/volumes"
Oct 30 19:13:32.605711 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:32.605694    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="edd86ca1-466c-4e45-8998-61d7e94fae7d" path="/var/lib/kubelet/pods/edd86ca1-466c-4e45-8998-61d7e94fae7d/volumes"
Oct 30 19:13:35.531303 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:13:35.531266    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:13:59.603831 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:13:59.603798    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9"]
Oct 30 19:14:02.078440 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.078405    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"]
Oct 30 19:14:02.078961 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.078946    4544 cpu_manager.go:414] "containerMap: removing container" podUID="b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" containerName="istio-proxy"
Oct 30 19:14:02.079008 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.078962    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" containerName="istio-proxy"
Oct 30 19:14:02.079008 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.078975    4544 cpu_manager.go:414] "containerMap: removing container" podUID="b1badcf1-f26a-4a5e-a068-678c9971e21c" containerName="istio-proxy"
Oct 30 19:14:02.079008 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.078983    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1badcf1-f26a-4a5e-a068-678c9971e21c" containerName="istio-proxy"
Oct 30 19:14:02.079008 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.078992    4544 cpu_manager.go:414] "containerMap: removing container" podUID="edd86ca1-466c-4e45-8998-61d7e94fae7d" containerName="istio-proxy"
Oct 30 19:14:02.079008 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.078997    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="edd86ca1-466c-4e45-8998-61d7e94fae7d" containerName="istio-proxy"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079015    4544 cpu_manager.go:414] "containerMap: removing container" podUID="e14048da-861d-4530-851e-8417b6cbf8bb" containerName="istio-proxy"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079019    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e14048da-861d-4530-851e-8417b6cbf8bb" containerName="istio-proxy"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079027    4544 cpu_manager.go:414] "containerMap: removing container" podUID="5ccae868-24e4-456f-ad9a-58c17d714b4e" containerName="istio-proxy"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079032    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ccae868-24e4-456f-ad9a-58c17d714b4e" containerName="istio-proxy"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079044    4544 cpu_manager.go:414] "containerMap: removing container" podUID="7482faa1-c2d0-4c2d-8998-b71343b0a141" containerName="payload-pre-processing"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079049    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7482faa1-c2d0-4c2d-8998-b71343b0a141" containerName="payload-pre-processing"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079055    4544 cpu_manager.go:414] "containerMap: removing container" podUID="b1badcf1-f26a-4a5e-a068-678c9971e21c" containerName="istio-proxy"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079059    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1badcf1-f26a-4a5e-a068-678c9971e21c" containerName="istio-proxy"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079070    4544 cpu_manager.go:414] "containerMap: removing container" podUID="0456a5cd-81b2-4dd1-9e36-a51808065ad0" containerName="istio-proxy"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079074    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0456a5cd-81b2-4dd1-9e36-a51808065ad0" containerName="istio-proxy"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079083    4544 cpu_manager.go:414] "containerMap: removing container" podUID="f117897d-7353-4281-909c-a475e208036d" containerName="payload-processing"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079087    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f117897d-7353-4281-909c-a475e208036d" containerName="payload-processing"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079101    4544 cpu_manager.go:414] "containerMap: removing container" podUID="0686c324-b270-432b-9dbb-8d6ad419a957" containerName="istio-proxy"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079106    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0686c324-b270-432b-9dbb-8d6ad419a957" containerName="istio-proxy"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079113    4544 cpu_manager.go:414] "containerMap: removing container" podUID="0456a5cd-81b2-4dd1-9e36-a51808065ad0" containerName="istio-proxy"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079118    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0456a5cd-81b2-4dd1-9e36-a51808065ad0" containerName="istio-proxy"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079124    4544 cpu_manager.go:414] "containerMap: removing container" podUID="18c8ec61-e79e-4370-a890-477e9ba68f10" containerName="payload-pre-processing"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079128    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18c8ec61-e79e-4370-a890-477e9ba68f10" containerName="payload-pre-processing"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079135    4544 cpu_manager.go:414] "containerMap: removing container" podUID="a5dea2af-18e6-458e-9e34-561bb06ba3f6" containerName="payload-processing"
Oct 30 19:14:02.079166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079139    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5dea2af-18e6-458e-9e34-561bb06ba3f6" containerName="payload-processing"
Oct 30 19:14:02.079944 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079214    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ccae868-24e4-456f-ad9a-58c17d714b4e" containerName="istio-proxy"
Oct 30 19:14:02.079944 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079222    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="7482faa1-c2d0-4c2d-8998-b71343b0a141" containerName="payload-pre-processing"
Oct 30 19:14:02.079944 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079230    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1badcf1-f26a-4a5e-a068-678c9971e21c" containerName="istio-proxy"
Oct 30 19:14:02.079944 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079238    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="0456a5cd-81b2-4dd1-9e36-a51808065ad0" containerName="istio-proxy"
Oct 30 19:14:02.079944 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079244    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="f117897d-7353-4281-909c-a475e208036d" containerName="payload-processing"
Oct 30 19:14:02.079944 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079250    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="0686c324-b270-432b-9dbb-8d6ad419a957" containerName="istio-proxy"
Oct 30 19:14:02.079944 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079254    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="0456a5cd-81b2-4dd1-9e36-a51808065ad0" containerName="istio-proxy"
Oct 30 19:14:02.079944 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079260    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="18c8ec61-e79e-4370-a890-477e9ba68f10" containerName="payload-pre-processing"
Oct 30 19:14:02.079944 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079266    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5dea2af-18e6-458e-9e34-561bb06ba3f6" containerName="payload-processing"
Oct 30 19:14:02.079944 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079271    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="b4ac89bb-06f0-4b1d-a5f2-98a3d59f37b4" containerName="istio-proxy"
Oct 30 19:14:02.079944 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079276    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="edd86ca1-466c-4e45-8998-61d7e94fae7d" containerName="istio-proxy"
Oct 30 19:14:02.079944 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.079283    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="e14048da-861d-4530-851e-8417b6cbf8bb" containerName="istio-proxy"
Oct 30 19:14:02.084526 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.084508    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"
Oct 30 19:14:02.193203 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.193169    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-f75d5f-serving-cert\""
Oct 30 19:14:02.196819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.196792    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"]
Oct 30 19:14:02.261443 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.261413    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22626496-4847-4999-a9b4-97b374162f1c-metrics-certs\") pod \"maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k\" (UID: \"22626496-4847-4999-a9b4-97b374162f1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"
Oct 30 19:14:02.261654 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.261562    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22626496-4847-4999-a9b4-97b374162f1c-maas-api-tls\") pod \"maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k\" (UID: \"22626496-4847-4999-a9b4-97b374162f1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"
Oct 30 19:14:02.261654 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.261628    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22626496-4847-4999-a9b4-97b374162f1c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k\" (UID: \"22626496-4847-4999-a9b4-97b374162f1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"
Oct 30 19:14:02.261798 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.261785    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2phd\" (UniqueName: \"kubernetes.io/projected/22626496-4847-4999-a9b4-97b374162f1c-kube-api-access-k2phd\") pod \"maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k\" (UID: \"22626496-4847-4999-a9b4-97b374162f1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"
Oct 30 19:14:02.363154 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.363067    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2phd\" (UniqueName: \"kubernetes.io/projected/22626496-4847-4999-a9b4-97b374162f1c-kube-api-access-k2phd\") pod \"maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k\" (UID: \"22626496-4847-4999-a9b4-97b374162f1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"
Oct 30 19:14:02.363328 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.363188    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22626496-4847-4999-a9b4-97b374162f1c-metrics-certs\") pod \"maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k\" (UID: \"22626496-4847-4999-a9b4-97b374162f1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"
Oct 30 19:14:02.363328 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.363291    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22626496-4847-4999-a9b4-97b374162f1c-maas-api-tls\") pod \"maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k\" (UID: \"22626496-4847-4999-a9b4-97b374162f1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"
Oct 30 19:14:02.363328 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.363318    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22626496-4847-4999-a9b4-97b374162f1c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k\" (UID: \"22626496-4847-4999-a9b4-97b374162f1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"
Oct 30 19:14:02.364037 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.364011    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22626496-4847-4999-a9b4-97b374162f1c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k\" (UID: \"22626496-4847-4999-a9b4-97b374162f1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"
Oct 30 19:14:02.367113 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.367087    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22626496-4847-4999-a9b4-97b374162f1c-metrics-certs\") pod \"maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k\" (UID: \"22626496-4847-4999-a9b4-97b374162f1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"
Oct 30 19:14:02.367357 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.367334    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22626496-4847-4999-a9b4-97b374162f1c-maas-api-tls\") pod \"maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k\" (UID: \"22626496-4847-4999-a9b4-97b374162f1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"
Oct 30 19:14:02.422460 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.422430    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2phd\" (UniqueName: \"kubernetes.io/projected/22626496-4847-4999-a9b4-97b374162f1c-kube-api-access-k2phd\") pod \"maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k\" (UID: \"22626496-4847-4999-a9b4-97b374162f1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"
Oct 30 19:14:02.479167 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.470260    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"]
Oct 30 19:14:02.479167 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.470546    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n" podUID="b58b94c2-d465-4560-aac1-1393be5c6343" containerName="istio-proxy" containerID="cri-o://f812aaf9a69d22ff1c9977a73e78afbbbfb8adde6eb754ced8bdf21f5edb830b" gracePeriod=30
Oct 30 19:14:02.698416 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:02.698319    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"
Oct 30 19:14:03.071951 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:14:03.071905    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22626496_4847_4999_a9b4_97b374162f1c.slice/crio-3d1e0926f1ee6a0e31c64d61e01b32d0acee50dc91f084f8ba15420b71b261c9 WatchSource:0}: Error finding container 3d1e0926f1ee6a0e31c64d61e01b32d0acee50dc91f084f8ba15420b71b261c9: Status 404 returned error can't find the container with id 3d1e0926f1ee6a0e31c64d61e01b32d0acee50dc91f084f8ba15420b71b261c9
Oct 30 19:14:03.072528 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:03.072500    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"]
Oct 30 19:14:03.073437 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:03.073416    4544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:14:03.567358 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:03.567316    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k" event={"ID":"22626496-4847-4999-a9b4-97b374162f1c","Type":"ContainerStarted","Data":"3d1e0926f1ee6a0e31c64d61e01b32d0acee50dc91f084f8ba15420b71b261c9"}
Oct 30 19:14:04.576695 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:04.576654    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k" event={"ID":"22626496-4847-4999-a9b4-97b374162f1c","Type":"ContainerStarted","Data":"7b9cc64ef8e1bafbcba74ae631df78b1d049d08fd3f4e637825ea5f85074772e"}
Oct 30 19:14:04.577286 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:04.576745    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"
Oct 30 19:14:04.604172 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:04.604123    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9" podUID="7bd6f01b-a7cb-48b3-be4c-0c386174170d" containerName="payload-processing" containerID="cri-o://e7f2ff83b2c4e24613849fb4ee29138569e8b4287ce7d282b5ee1b16e68ea396" gracePeriod=25
Oct 30 19:14:04.694345 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:14:04.694317    4544 logging.go:55] [core] [Channel #2282 SubChannel #2283]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.92:50052", ServerName: "10.133.0.92:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.92:50052: connect: connection refused"
Oct 30 19:14:04.697611 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:04.697543    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k" podStartSLOduration=1.7575597379999999 podStartE2EDuration="2.697525453s" totalImagesPullingTime="939.965715ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:04.694180861 +0000 UTC m=+2242.220055958" watchObservedRunningTime="2026-10-30 19:14:04.697525453 +0000 UTC m=+2242.223400550"
Oct 30 19:14:04.919963 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:04.919937    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9"
Oct 30 19:14:05.114626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:05.114157    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bd6f01b-a7cb-48b3-be4c-0c386174170d-kube-api-access-m58ds\" (UniqueName: \"kubernetes.io/projected/7bd6f01b-a7cb-48b3-be4c-0c386174170d-kube-api-access-m58ds\") pod \"7bd6f01b-a7cb-48b3-be4c-0c386174170d\" (UID: \"7bd6f01b-a7cb-48b3-be4c-0c386174170d\") "
Oct 30 19:14:05.114626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:05.114260    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7bd6f01b-a7cb-48b3-be4c-0c386174170d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bd6f01b-a7cb-48b3-be4c-0c386174170d-plugins-config-volume\") pod \"7bd6f01b-a7cb-48b3-be4c-0c386174170d\" (UID: \"7bd6f01b-a7cb-48b3-be4c-0c386174170d\") "
Oct 30 19:14:05.125626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:05.123770    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bd6f01b-a7cb-48b3-be4c-0c386174170d-plugins-config-volume" pod "7bd6f01b-a7cb-48b3-be4c-0c386174170d" (UID: "7bd6f01b-a7cb-48b3-be4c-0c386174170d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:14:05.161424 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:05.160724    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bd6f01b-a7cb-48b3-be4c-0c386174170d-kube-api-access-m58ds" pod "7bd6f01b-a7cb-48b3-be4c-0c386174170d" (UID: "7bd6f01b-a7cb-48b3-be4c-0c386174170d"). InnerVolumeSpecName "kube-api-access-m58ds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:05.216065 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:05.216034    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m58ds\" (UniqueName: \"kubernetes.io/projected/7bd6f01b-a7cb-48b3-be4c-0c386174170d-kube-api-access-m58ds\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:05.216065 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:05.216062    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bd6f01b-a7cb-48b3-be4c-0c386174170d-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:05.532648 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:14:05.532597    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:14:05.584352 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:05.584315    4544 generic.go:356] "Generic (PLEG): container finished" podID="7bd6f01b-a7cb-48b3-be4c-0c386174170d" containerID="e7f2ff83b2c4e24613849fb4ee29138569e8b4287ce7d282b5ee1b16e68ea396" exitCode=0
Oct 30 19:14:05.584789 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:05.584371    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9"
Oct 30 19:14:05.584789 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:05.584403    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9" event={"ID":"7bd6f01b-a7cb-48b3-be4c-0c386174170d","Type":"ContainerDied","Data":"e7f2ff83b2c4e24613849fb4ee29138569e8b4287ce7d282b5ee1b16e68ea396"}
Oct 30 19:14:05.584789 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:05.584473    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9" event={"ID":"7bd6f01b-a7cb-48b3-be4c-0c386174170d","Type":"ContainerDied","Data":"b4d88a305746905d25a4cac8672483306bc88ec2de297d59c3d83372e23b355a"}
Oct 30 19:14:05.584789 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:05.584505    4544 scope.go:122] "RemoveContainer" containerID="e7f2ff83b2c4e24613849fb4ee29138569e8b4287ce7d282b5ee1b16e68ea396"
Oct 30 19:14:05.596567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:05.596551    4544 scope.go:122] "RemoveContainer" containerID="e7f2ff83b2c4e24613849fb4ee29138569e8b4287ce7d282b5ee1b16e68ea396"
Oct 30 19:14:05.596863 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:14:05.596844    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7f2ff83b2c4e24613849fb4ee29138569e8b4287ce7d282b5ee1b16e68ea396\": container with ID starting with e7f2ff83b2c4e24613849fb4ee29138569e8b4287ce7d282b5ee1b16e68ea396 not found: ID does not exist" containerID="e7f2ff83b2c4e24613849fb4ee29138569e8b4287ce7d282b5ee1b16e68ea396"
Oct 30 19:14:05.596943 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:05.596871    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7f2ff83b2c4e24613849fb4ee29138569e8b4287ce7d282b5ee1b16e68ea396"} err="failed to get container status \"e7f2ff83b2c4e24613849fb4ee29138569e8b4287ce7d282b5ee1b16e68ea396\": rpc error: code = NotFound desc = could not find container \"e7f2ff83b2c4e24613849fb4ee29138569e8b4287ce7d282b5ee1b16e68ea396\": container with ID starting with e7f2ff83b2c4e24613849fb4ee29138569e8b4287ce7d282b5ee1b16e68ea396 not found: ID does not exist"
Oct 30 19:14:05.694732 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:05.694687    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9" podUID="7bd6f01b-a7cb-48b3-be4c-0c386174170d" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.92:50052\" within 1s: context deadline exceeded"
Oct 30 19:14:05.740815 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:05.740775    4544 patch_prober.go:29] interesting pod/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:14:05.740984 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:05.740827    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n" podUID="b58b94c2-d465-4560-aac1-1393be5c6343" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:14:05.836501 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:05.836382    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9"]
Oct 30 19:14:05.862930 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:05.862898    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-091e8f-dcc96f6f4-xv8x9"]
Oct 30 19:14:06.603455 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:06.603422    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7bd6f01b-a7cb-48b3-be4c-0c386174170d" path="/var/lib/kubelet/pods/7bd6f01b-a7cb-48b3-be4c-0c386174170d/volumes"
Oct 30 19:14:07.603397 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.603202    4544 generic.go:356] "Generic (PLEG): container finished" podID="b58b94c2-d465-4560-aac1-1393be5c6343" containerID="f812aaf9a69d22ff1c9977a73e78afbbbfb8adde6eb754ced8bdf21f5edb830b" exitCode=0
Oct 30 19:14:07.603397 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.603274    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n" event={"ID":"b58b94c2-d465-4560-aac1-1393be5c6343","Type":"ContainerDied","Data":"f812aaf9a69d22ff1c9977a73e78afbbbfb8adde6eb754ced8bdf21f5edb830b"}
Oct 30 19:14:07.768011 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.767979    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:14:07.848087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.848048    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b58b94c2-d465-4560-aac1-1393be5c6343-kube-api-access-44qrb\" (UniqueName: \"kubernetes.io/projected/b58b94c2-d465-4560-aac1-1393be5c6343-kube-api-access-44qrb\") pod \"b58b94c2-d465-4560-aac1-1393be5c6343\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") "
Oct 30 19:14:07.848257 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.848114    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-istio-data\") pod \"b58b94c2-d465-4560-aac1-1393be5c6343\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") "
Oct 30 19:14:07.848257 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.848138    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-istio-envoy\") pod \"b58b94c2-d465-4560-aac1-1393be5c6343\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") "
Oct 30 19:14:07.848257 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.848183    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b58b94c2-d465-4560-aac1-1393be5c6343-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b58b94c2-d465-4560-aac1-1393be5c6343-istiod-ca-cert\") pod \"b58b94c2-d465-4560-aac1-1393be5c6343\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") "
Oct 30 19:14:07.848257 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.848238    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-credential-socket\") pod \"b58b94c2-d465-4560-aac1-1393be5c6343\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") "
Oct 30 19:14:07.848466 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.848296    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b58b94c2-d465-4560-aac1-1393be5c6343-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b58b94c2-d465-4560-aac1-1393be5c6343-istio-podinfo\") pod \"b58b94c2-d465-4560-aac1-1393be5c6343\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") "
Oct 30 19:14:07.848466 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.848329    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-workload-socket\") pod \"b58b94c2-d465-4560-aac1-1393be5c6343\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") "
Oct 30 19:14:07.848466 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.848363    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-workload-certs\") pod \"b58b94c2-d465-4560-aac1-1393be5c6343\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") "
Oct 30 19:14:07.848643 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.848500    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-istio-data" pod "b58b94c2-d465-4560-aac1-1393be5c6343" (UID: "b58b94c2-d465-4560-aac1-1393be5c6343"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:07.848643 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.848615    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-workload-socket" pod "b58b94c2-d465-4560-aac1-1393be5c6343" (UID: "b58b94c2-d465-4560-aac1-1393be5c6343"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:07.848750 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.848686    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-credential-socket" pod "b58b94c2-d465-4560-aac1-1393be5c6343" (UID: "b58b94c2-d465-4560-aac1-1393be5c6343"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:07.848750 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.848726    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-workload-certs" pod "b58b94c2-d465-4560-aac1-1393be5c6343" (UID: "b58b94c2-d465-4560-aac1-1393be5c6343"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:07.848981 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.848953    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b58b94c2-d465-4560-aac1-1393be5c6343-istiod-ca-cert" pod "b58b94c2-d465-4560-aac1-1393be5c6343" (UID: "b58b94c2-d465-4560-aac1-1393be5c6343"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:14:07.849117 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.849019    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:07.849117 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.849041    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:07.849117 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.849056    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:07.849117 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.849071    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:07.850463 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.850433    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b58b94c2-d465-4560-aac1-1393be5c6343-kube-api-access-44qrb" pod "b58b94c2-d465-4560-aac1-1393be5c6343" (UID: "b58b94c2-d465-4560-aac1-1393be5c6343"). InnerVolumeSpecName "kube-api-access-44qrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:07.850463 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.850458    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-istio-envoy" pod "b58b94c2-d465-4560-aac1-1393be5c6343" (UID: "b58b94c2-d465-4560-aac1-1393be5c6343"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:07.850591 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.850456    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b58b94c2-d465-4560-aac1-1393be5c6343-istio-podinfo" pod "b58b94c2-d465-4560-aac1-1393be5c6343" (UID: "b58b94c2-d465-4560-aac1-1393be5c6343"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:14:07.950269 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.950237    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b58b94c2-d465-4560-aac1-1393be5c6343-istio-token\" (UniqueName: \"kubernetes.io/projected/b58b94c2-d465-4560-aac1-1393be5c6343-istio-token\") pod \"b58b94c2-d465-4560-aac1-1393be5c6343\" (UID: \"b58b94c2-d465-4560-aac1-1393be5c6343\") "
Oct 30 19:14:07.950798 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.950774    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44qrb\" (UniqueName: \"kubernetes.io/projected/b58b94c2-d465-4560-aac1-1393be5c6343-kube-api-access-44qrb\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:07.950902 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.950805    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b58b94c2-d465-4560-aac1-1393be5c6343-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:07.950902 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.950820    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b58b94c2-d465-4560-aac1-1393be5c6343-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:07.950902 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.950834    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b58b94c2-d465-4560-aac1-1393be5c6343-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:07.952311 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:07.952288    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b58b94c2-d465-4560-aac1-1393be5c6343-istio-token" pod "b58b94c2-d465-4560-aac1-1393be5c6343" (UID: "b58b94c2-d465-4560-aac1-1393be5c6343"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:08.051548 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:08.051474    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b58b94c2-d465-4560-aac1-1393be5c6343-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:08.611100 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:08.611065    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"
Oct 30 19:14:08.611100 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:08.611083    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n" event={"ID":"b58b94c2-d465-4560-aac1-1393be5c6343","Type":"ContainerDied","Data":"bbf47ad549363129eedd24a9dcd642438a7511916096b82b5b0ef02d6b0074e9"}
Oct 30 19:14:08.611744 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:08.611152    4544 scope.go:122] "RemoveContainer" containerID="f812aaf9a69d22ff1c9977a73e78afbbbfb8adde6eb754ced8bdf21f5edb830b"
Oct 30 19:14:08.780283 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:08.780198    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"]
Oct 30 19:14:08.828182 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:08.828152    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-29j2n"]
Oct 30 19:14:10.608034 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:10.608000    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b58b94c2-d465-4560-aac1-1393be5c6343" path="/var/lib/kubelet/pods/b58b94c2-d465-4560-aac1-1393be5c6343/volumes"
Oct 30 19:14:10.608419 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:10.608346    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"
Oct 30 19:14:18.305946 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:18.305899    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl"]
Oct 30 19:14:18.417215 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:18.417179    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6"]
Oct 30 19:14:23.306683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.306638    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl" podUID="2f6a351b-2333-4f27-a6d6-957830a556a4" containerName="payload-processing" containerID="cri-o://1006f9b149d26963b54b81fe218f9fc05fd3106e1c0311a639cd03fcc2356414" gracePeriod=25
Oct 30 19:14:23.418232 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.418166    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6" podUID="21d57f6e-955d-4b8a-bad4-45c9cfdd80ba" containerName="payload-pre-processing" containerID="cri-o://92bd3a3d0a42d353d34ccf30f45b902f98c77220797262e9aba3b175b868a200" gracePeriod=25
Oct 30 19:14:23.674871 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.671131    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl"
Oct 30 19:14:23.757002 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.756977    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6"
Oct 30 19:14:23.768977 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.768749    4544 generic.go:356] "Generic (PLEG): container finished" podID="21d57f6e-955d-4b8a-bad4-45c9cfdd80ba" containerID="92bd3a3d0a42d353d34ccf30f45b902f98c77220797262e9aba3b175b868a200" exitCode=0
Oct 30 19:14:23.768977 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.768799    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6"
Oct 30 19:14:23.768977 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.768804    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6" event={"ID":"21d57f6e-955d-4b8a-bad4-45c9cfdd80ba","Type":"ContainerDied","Data":"92bd3a3d0a42d353d34ccf30f45b902f98c77220797262e9aba3b175b868a200"}
Oct 30 19:14:23.768977 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.768855    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6" event={"ID":"21d57f6e-955d-4b8a-bad4-45c9cfdd80ba","Type":"ContainerDied","Data":"13f06ab68baadd604f381de9ef3460a2ec122886f00eb341e31134ddb27dd145"}
Oct 30 19:14:23.768977 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.768879    4544 scope.go:122] "RemoveContainer" containerID="92bd3a3d0a42d353d34ccf30f45b902f98c77220797262e9aba3b175b868a200"
Oct 30 19:14:23.770565 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.770542    4544 generic.go:356] "Generic (PLEG): container finished" podID="2f6a351b-2333-4f27-a6d6-957830a556a4" containerID="1006f9b149d26963b54b81fe218f9fc05fd3106e1c0311a639cd03fcc2356414" exitCode=0
Oct 30 19:14:23.770721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.770589    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl"
Oct 30 19:14:23.770721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.770642    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl" event={"ID":"2f6a351b-2333-4f27-a6d6-957830a556a4","Type":"ContainerDied","Data":"1006f9b149d26963b54b81fe218f9fc05fd3106e1c0311a639cd03fcc2356414"}
Oct 30 19:14:23.770721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.770700    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl" event={"ID":"2f6a351b-2333-4f27-a6d6-957830a556a4","Type":"ContainerDied","Data":"227b8b00265711119969b085879ab47c86f94858357c0dc6c2d254204f99646b"}
Oct 30 19:14:23.777024 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.776999    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f6a351b-2333-4f27-a6d6-957830a556a4-kube-api-access-ctfjz\" (UniqueName: \"kubernetes.io/projected/2f6a351b-2333-4f27-a6d6-957830a556a4-kube-api-access-ctfjz\") pod \"2f6a351b-2333-4f27-a6d6-957830a556a4\" (UID: \"2f6a351b-2333-4f27-a6d6-957830a556a4\") "
Oct 30 19:14:23.777154 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.777139    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f6a351b-2333-4f27-a6d6-957830a556a4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f6a351b-2333-4f27-a6d6-957830a556a4-plugins-config-volume\") pod \"2f6a351b-2333-4f27-a6d6-957830a556a4\" (UID: \"2f6a351b-2333-4f27-a6d6-957830a556a4\") "
Oct 30 19:14:23.777636 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.777587    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f6a351b-2333-4f27-a6d6-957830a556a4-plugins-config-volume" pod "2f6a351b-2333-4f27-a6d6-957830a556a4" (UID: "2f6a351b-2333-4f27-a6d6-957830a556a4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:14:23.778751 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.778656    4544 scope.go:122] "RemoveContainer" containerID="92bd3a3d0a42d353d34ccf30f45b902f98c77220797262e9aba3b175b868a200"
Oct 30 19:14:23.778981 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:14:23.778957    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92bd3a3d0a42d353d34ccf30f45b902f98c77220797262e9aba3b175b868a200\": container with ID starting with 92bd3a3d0a42d353d34ccf30f45b902f98c77220797262e9aba3b175b868a200 not found: ID does not exist" containerID="92bd3a3d0a42d353d34ccf30f45b902f98c77220797262e9aba3b175b868a200"
Oct 30 19:14:23.779067 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.778992    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92bd3a3d0a42d353d34ccf30f45b902f98c77220797262e9aba3b175b868a200"} err="failed to get container status \"92bd3a3d0a42d353d34ccf30f45b902f98c77220797262e9aba3b175b868a200\": rpc error: code = NotFound desc = could not find container \"92bd3a3d0a42d353d34ccf30f45b902f98c77220797262e9aba3b175b868a200\": container with ID starting with 92bd3a3d0a42d353d34ccf30f45b902f98c77220797262e9aba3b175b868a200 not found: ID does not exist"
Oct 30 19:14:23.779067 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.779012    4544 scope.go:122] "RemoveContainer" containerID="1006f9b149d26963b54b81fe218f9fc05fd3106e1c0311a639cd03fcc2356414"
Oct 30 19:14:23.779279 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.779255    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f6a351b-2333-4f27-a6d6-957830a556a4-kube-api-access-ctfjz" pod "2f6a351b-2333-4f27-a6d6-957830a556a4" (UID: "2f6a351b-2333-4f27-a6d6-957830a556a4"). InnerVolumeSpecName "kube-api-access-ctfjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:23.790022 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.789999    4544 scope.go:122] "RemoveContainer" containerID="1006f9b149d26963b54b81fe218f9fc05fd3106e1c0311a639cd03fcc2356414"
Oct 30 19:14:23.790348 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:14:23.790325    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1006f9b149d26963b54b81fe218f9fc05fd3106e1c0311a639cd03fcc2356414\": container with ID starting with 1006f9b149d26963b54b81fe218f9fc05fd3106e1c0311a639cd03fcc2356414 not found: ID does not exist" containerID="1006f9b149d26963b54b81fe218f9fc05fd3106e1c0311a639cd03fcc2356414"
Oct 30 19:14:23.790423 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.790359    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1006f9b149d26963b54b81fe218f9fc05fd3106e1c0311a639cd03fcc2356414"} err="failed to get container status \"1006f9b149d26963b54b81fe218f9fc05fd3106e1c0311a639cd03fcc2356414\": rpc error: code = NotFound desc = could not find container \"1006f9b149d26963b54b81fe218f9fc05fd3106e1c0311a639cd03fcc2356414\": container with ID starting with 1006f9b149d26963b54b81fe218f9fc05fd3106e1c0311a639cd03fcc2356414 not found: ID does not exist"
Oct 30 19:14:23.878382 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.878350    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21d57f6e-955d-4b8a-bad4-45c9cfdd80ba-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d57f6e-955d-4b8a-bad4-45c9cfdd80ba-plugins-config-volume\") pod \"21d57f6e-955d-4b8a-bad4-45c9cfdd80ba\" (UID: \"21d57f6e-955d-4b8a-bad4-45c9cfdd80ba\") "
Oct 30 19:14:23.878581 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.878527    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21d57f6e-955d-4b8a-bad4-45c9cfdd80ba-kube-api-access-qqt9d\" (UniqueName: \"kubernetes.io/projected/21d57f6e-955d-4b8a-bad4-45c9cfdd80ba-kube-api-access-qqt9d\") pod \"21d57f6e-955d-4b8a-bad4-45c9cfdd80ba\" (UID: \"21d57f6e-955d-4b8a-bad4-45c9cfdd80ba\") "
Oct 30 19:14:23.878789 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.878763    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21d57f6e-955d-4b8a-bad4-45c9cfdd80ba-plugins-config-volume" pod "21d57f6e-955d-4b8a-bad4-45c9cfdd80ba" (UID: "21d57f6e-955d-4b8a-bad4-45c9cfdd80ba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:14:23.878989 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.878973    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d57f6e-955d-4b8a-bad4-45c9cfdd80ba-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:23.879051 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.878995    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctfjz\" (UniqueName: \"kubernetes.io/projected/2f6a351b-2333-4f27-a6d6-957830a556a4-kube-api-access-ctfjz\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:23.879051 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.879010    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f6a351b-2333-4f27-a6d6-957830a556a4-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:23.880703 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.880666    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21d57f6e-955d-4b8a-bad4-45c9cfdd80ba-kube-api-access-qqt9d" pod "21d57f6e-955d-4b8a-bad4-45c9cfdd80ba" (UID: "21d57f6e-955d-4b8a-bad4-45c9cfdd80ba"). InnerVolumeSpecName "kube-api-access-qqt9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:23.984450 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:23.984364    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqt9d\" (UniqueName: \"kubernetes.io/projected/21d57f6e-955d-4b8a-bad4-45c9cfdd80ba-kube-api-access-qqt9d\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:24.190031 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:24.189996    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6"]
Oct 30 19:14:24.193978 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:24.193953    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0f3f2f-b7bd7d487-t2ct6"]
Oct 30 19:14:24.303938 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:24.303895    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl"]
Oct 30 19:14:24.306956 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:24.306934    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w6-0f3f2f-7565c854bd-fc9xl"]
Oct 30 19:14:24.619569 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:24.619477    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21d57f6e-955d-4b8a-bad4-45c9cfdd80ba" path="/var/lib/kubelet/pods/21d57f6e-955d-4b8a-bad4-45c9cfdd80ba/volumes"
Oct 30 19:14:24.620016 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:24.619997    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f6a351b-2333-4f27-a6d6-957830a556a4" path="/var/lib/kubelet/pods/2f6a351b-2333-4f27-a6d6-957830a556a4/volumes"
Oct 30 19:14:33.406805 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.406763    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"]
Oct 30 19:14:33.407511 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.407488    4544 cpu_manager.go:414] "containerMap: removing container" podUID="2f6a351b-2333-4f27-a6d6-957830a556a4" containerName="payload-processing"
Oct 30 19:14:33.407655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.407512    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f6a351b-2333-4f27-a6d6-957830a556a4" containerName="payload-processing"
Oct 30 19:14:33.407655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.407551    4544 cpu_manager.go:414] "containerMap: removing container" podUID="21d57f6e-955d-4b8a-bad4-45c9cfdd80ba" containerName="payload-pre-processing"
Oct 30 19:14:33.407655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.407559    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21d57f6e-955d-4b8a-bad4-45c9cfdd80ba" containerName="payload-pre-processing"
Oct 30 19:14:33.407655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.407572    4544 cpu_manager.go:414] "containerMap: removing container" podUID="7bd6f01b-a7cb-48b3-be4c-0c386174170d" containerName="payload-processing"
Oct 30 19:14:33.407655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.407580    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7bd6f01b-a7cb-48b3-be4c-0c386174170d" containerName="payload-processing"
Oct 30 19:14:33.407655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.407597    4544 cpu_manager.go:414] "containerMap: removing container" podUID="b58b94c2-d465-4560-aac1-1393be5c6343" containerName="istio-proxy"
Oct 30 19:14:33.407655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.407618    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b58b94c2-d465-4560-aac1-1393be5c6343" containerName="istio-proxy"
Oct 30 19:14:33.408005 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.407771    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="21d57f6e-955d-4b8a-bad4-45c9cfdd80ba" containerName="payload-pre-processing"
Oct 30 19:14:33.408005 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.407786    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="7bd6f01b-a7cb-48b3-be4c-0c386174170d" containerName="payload-processing"
Oct 30 19:14:33.408005 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.407799    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="b58b94c2-d465-4560-aac1-1393be5c6343" containerName="istio-proxy"
Oct 30 19:14:33.408005 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.407813    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1badcf1-f26a-4a5e-a068-678c9971e21c" containerName="istio-proxy"
Oct 30 19:14:33.408005 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.407829    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f6a351b-2333-4f27-a6d6-957830a556a4" containerName="payload-processing"
Oct 30 19:14:33.412591 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.412571    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.510959 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.510919    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psn5p\" (UniqueName: \"kubernetes.io/projected/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-kube-api-access-psn5p\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.511135 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.510984    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-podinfo\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.511135 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.511101    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-workload-socket\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.511270 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.511158    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-token\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.511270 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.511241    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istiod-ca-cert\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.511389 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.511282    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-data\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.511441 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.511384    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-envoy\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.511441 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.511418    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-credential-socket\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.511558 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.511445    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-workload-certs\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.612412 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.612376    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"]
Oct 30 19:14:33.612947 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.612919    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-data\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.613078 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.612990    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-envoy\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.613078 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.613019    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-credential-socket\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.613078 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.613048    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-workload-certs\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.613240 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.613149    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psn5p\" (UniqueName: \"kubernetes.io/projected/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-kube-api-access-psn5p\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.613240 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.613191    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-podinfo\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.613348 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.613268    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-workload-socket\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.613348 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.613325    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-token\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.613456 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.613375    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istiod-ca-cert\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.613456 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.613399    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-data\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.613556 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.613485    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-workload-certs\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.613811 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.613754    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-workload-socket\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.614142 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.614118    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istiod-ca-cert\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.614247 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.614185    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-credential-socket\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.615828 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.615799    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-envoy\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.616483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.616458    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-podinfo\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.673486 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.673414    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-token\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.673648 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.673509    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psn5p\" (UniqueName: \"kubernetes.io/projected/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-kube-api-access-psn5p\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:33.725809 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:33.725783    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:34.071038 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:34.070991    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"]
Oct 30 19:14:34.071248 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:34.071212    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb" podUID="b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" containerName="istio-proxy" containerID="cri-o://3c9e07561cde5edd626990a4690c7ce3593f36e0972f5d1fe09b2295c452c0db" gracePeriod=30
Oct 30 19:14:34.253211 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:34.246512    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"]
Oct 30 19:14:34.261772 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:34.261730    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:14:34.261890 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:34.261848    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:14:34.261957 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:34.261908    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:14:34.844856 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:34.844818    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj" event={"ID":"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95","Type":"ContainerStarted","Data":"03ba2e5b25ef574211f41d1aa8171ebf7d7a3aaa28d70885b9288298a6aab32e"}
Oct 30 19:14:34.845472 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:34.844868    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj" event={"ID":"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95","Type":"ContainerStarted","Data":"3ef897813bff3f1115797c52b3265261d457d486a59b110739f971ba37e2cc61"}
Oct 30 19:14:34.970626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:34.966007    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj" podStartSLOduration=1.9659869620000001 podStartE2EDuration="1.965986962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:34.956551636 +0000 UTC m=+2272.482426733" watchObservedRunningTime="2026-10-30 19:14:34.965986962 +0000 UTC m=+2272.491862064"
Oct 30 19:14:35.531401 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:14:35.531361    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:14:35.726312 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:35.726274    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:35.728059 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:35.728031    4544 patch_prober.go:29] interesting pod/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Oct 30 19:14:35.728202 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:35.728098    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj" podUID="6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Oct 30 19:14:36.726291 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:36.726259    4544 patch_prober.go:29] interesting pod/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Oct 30 19:14:36.726711 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:36.726315    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj" podUID="6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Oct 30 19:14:36.937082 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:36.937045    4544 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:14:36.937283 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:36.937098    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb" podUID="b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:14:37.726547 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:37.726511    4544 patch_prober.go:29] interesting pod/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Oct 30 19:14:37.727049 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:37.726570    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj" podUID="6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Oct 30 19:14:38.726620 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:38.726575    4544 patch_prober.go:29] interesting pod/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Oct 30 19:14:38.727116 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:38.726645    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj" podUID="6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Oct 30 19:14:39.323523 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.323492    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:14:39.424572 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.424484    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-token\" (UniqueName: \"kubernetes.io/projected/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-token\") pod \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") "
Oct 30 19:14:39.424572 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.424541    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-workload-certs\") pod \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") "
Oct 30 19:14:39.424836 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.424590    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-envoy\") pod \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") "
Oct 30 19:14:39.424836 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.424649    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-workload-socket\") pod \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") "
Oct 30 19:14:39.424836 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.424669    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-kube-api-access-8p2gx\" (UniqueName: \"kubernetes.io/projected/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-kube-api-access-8p2gx\") pod \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") "
Oct 30 19:14:39.424836 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.424728    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istiod-ca-cert\") pod \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") "
Oct 30 19:14:39.424836 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.424814    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-podinfo\") pod \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") "
Oct 30 19:14:39.425114 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.424896    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-credential-socket\") pod \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") "
Oct 30 19:14:39.425114 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.424901    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-workload-certs" pod "b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" (UID: "b2e66881-cd8a-4bf9-9f15-0ced9d94a3db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:39.425114 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.424950    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-data\") pod \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\" (UID: \"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db\") "
Oct 30 19:14:39.425114 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.424947    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-workload-socket" pod "b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" (UID: "b2e66881-cd8a-4bf9-9f15-0ced9d94a3db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:39.425317 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.425210    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istiod-ca-cert" pod "b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" (UID: "b2e66881-cd8a-4bf9-9f15-0ced9d94a3db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:14:39.425317 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.425214    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-credential-socket" pod "b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" (UID: "b2e66881-cd8a-4bf9-9f15-0ced9d94a3db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:39.425629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.425584    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:39.425731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.425645    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:39.425731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.425660    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:39.425731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.425673    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:39.425731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.425586    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-data" pod "b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" (UID: "b2e66881-cd8a-4bf9-9f15-0ced9d94a3db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:39.427386 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.427360    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-envoy" pod "b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" (UID: "b2e66881-cd8a-4bf9-9f15-0ced9d94a3db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:39.427670 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.427648    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-kube-api-access-8p2gx" pod "b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" (UID: "b2e66881-cd8a-4bf9-9f15-0ced9d94a3db"). InnerVolumeSpecName "kube-api-access-8p2gx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:39.427830 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.427811    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-podinfo" pod "b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" (UID: "b2e66881-cd8a-4bf9-9f15-0ced9d94a3db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:14:39.428912 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.428887    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-token" pod "b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" (UID: "b2e66881-cd8a-4bf9-9f15-0ced9d94a3db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:39.527188 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.527153    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:39.527188 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.527186    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p2gx\" (UniqueName: \"kubernetes.io/projected/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-kube-api-access-8p2gx\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:39.527425 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.527198    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:39.527425 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.527207    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:39.527425 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.527216    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:39.726282 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.726201    4544 patch_prober.go:29] interesting pod/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused" start-of-body=
Oct 30 19:14:39.726437 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.726275    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj" podUID="6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.117:15021/healthz/ready\": dial tcp 10.133.0.117:15021: connect: connection refused"
Oct 30 19:14:39.904742 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.904707    4544 generic.go:356] "Generic (PLEG): container finished" podID="b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" containerID="3c9e07561cde5edd626990a4690c7ce3593f36e0972f5d1fe09b2295c452c0db" exitCode=0
Oct 30 19:14:39.905211 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.904750    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"
Oct 30 19:14:39.905211 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.904781    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb" event={"ID":"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db","Type":"ContainerDied","Data":"3c9e07561cde5edd626990a4690c7ce3593f36e0972f5d1fe09b2295c452c0db"}
Oct 30 19:14:39.905211 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.904834    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb" event={"ID":"b2e66881-cd8a-4bf9-9f15-0ced9d94a3db","Type":"ContainerDied","Data":"93667ee2b5788428c01cd697c8b7d4ca1e46bbdbaea53c0497ae9b7b38c06a39"}
Oct 30 19:14:39.905211 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.904869    4544 scope.go:122] "RemoveContainer" containerID="3c9e07561cde5edd626990a4690c7ce3593f36e0972f5d1fe09b2295c452c0db"
Oct 30 19:14:39.915049 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.915027    4544 scope.go:122] "RemoveContainer" containerID="0a71aabe4e79fb2504dd62e441b2e8649655cbade08bd02cea9011ad33f2c824"
Oct 30 19:14:39.923786 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.923771    4544 scope.go:122] "RemoveContainer" containerID="3c9e07561cde5edd626990a4690c7ce3593f36e0972f5d1fe09b2295c452c0db"
Oct 30 19:14:39.924056 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:14:39.924039    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c9e07561cde5edd626990a4690c7ce3593f36e0972f5d1fe09b2295c452c0db\": container with ID starting with 3c9e07561cde5edd626990a4690c7ce3593f36e0972f5d1fe09b2295c452c0db not found: ID does not exist" containerID="3c9e07561cde5edd626990a4690c7ce3593f36e0972f5d1fe09b2295c452c0db"
Oct 30 19:14:39.924097 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.924066    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c9e07561cde5edd626990a4690c7ce3593f36e0972f5d1fe09b2295c452c0db"} err="failed to get container status \"3c9e07561cde5edd626990a4690c7ce3593f36e0972f5d1fe09b2295c452c0db\": rpc error: code = NotFound desc = could not find container \"3c9e07561cde5edd626990a4690c7ce3593f36e0972f5d1fe09b2295c452c0db\": container with ID starting with 3c9e07561cde5edd626990a4690c7ce3593f36e0972f5d1fe09b2295c452c0db not found: ID does not exist"
Oct 30 19:14:39.924097 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.924083    4544 scope.go:122] "RemoveContainer" containerID="0a71aabe4e79fb2504dd62e441b2e8649655cbade08bd02cea9011ad33f2c824"
Oct 30 19:14:39.924298 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:14:39.924283    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a71aabe4e79fb2504dd62e441b2e8649655cbade08bd02cea9011ad33f2c824\": container with ID starting with 0a71aabe4e79fb2504dd62e441b2e8649655cbade08bd02cea9011ad33f2c824 not found: ID does not exist" containerID="0a71aabe4e79fb2504dd62e441b2e8649655cbade08bd02cea9011ad33f2c824"
Oct 30 19:14:39.924332 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:39.924303    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a71aabe4e79fb2504dd62e441b2e8649655cbade08bd02cea9011ad33f2c824"} err="failed to get container status \"0a71aabe4e79fb2504dd62e441b2e8649655cbade08bd02cea9011ad33f2c824\": rpc error: code = NotFound desc = could not find container \"0a71aabe4e79fb2504dd62e441b2e8649655cbade08bd02cea9011ad33f2c824\": container with ID starting with 0a71aabe4e79fb2504dd62e441b2e8649655cbade08bd02cea9011ad33f2c824 not found: ID does not exist"
Oct 30 19:14:40.082265 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:40.082216    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"]
Oct 30 19:14:40.085243 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:40.085218    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-0f3f2f-openshift-default-57b76d9d9f-2q6nb"]
Oct 30 19:14:40.606068 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:40.606030    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" path="/var/lib/kubelet/pods/b2e66881-cd8a-4bf9-9f15-0ced9d94a3db/volumes"
Oct 30 19:14:40.781870 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:40.781820    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:40.782468 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:40.782443    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:40.783721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:40.783703    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:14:48.491070 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.491033    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl"]
Oct 30 19:14:48.513748 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.513715    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"]
Oct 30 19:14:48.514300 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.514280    4544 cpu_manager.go:414] "containerMap: removing container" podUID="b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" containerName="istio-proxy"
Oct 30 19:14:48.514397 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.514301    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" containerName="istio-proxy"
Oct 30 19:14:48.514397 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.514319    4544 cpu_manager.go:414] "containerMap: removing container" podUID="b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" containerName="istio-proxy"
Oct 30 19:14:48.514397 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.514327    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" containerName="istio-proxy"
Oct 30 19:14:48.514554 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.514453    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" containerName="istio-proxy"
Oct 30 19:14:48.514554 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.514467    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2e66881-cd8a-4bf9-9f15-0ced9d94a3db" containerName="istio-proxy"
Oct 30 19:14:48.519735 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.519716    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.639772 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.639739    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"]
Oct 30 19:14:48.642427 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.642391    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhvw6\" (UniqueName: \"kubernetes.io/projected/e4dc6f73-5287-4f71-b368-6c0d605e9054-kube-api-access-dhvw6\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.642583 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.642493    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-data\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.642583 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.642540    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4dc6f73-5287-4f71-b368-6c0d605e9054-istiod-ca-cert\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.642731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.642667    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-credential-socket\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.642731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.642702    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-workload-socket\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.642799 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.642749    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-podinfo\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.642834 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.642820    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-workload-certs\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.642913 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.642897    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-token\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.642984 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.642969    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-envoy\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.744396 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.744308    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-credential-socket\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.744396 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.744352    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-workload-socket\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.744396 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.744376    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-podinfo\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.744690 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.744431    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-workload-certs\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.744690 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.744487    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-token\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.744690 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.744531    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-envoy\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.744690 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.744573    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhvw6\" (UniqueName: \"kubernetes.io/projected/e4dc6f73-5287-4f71-b368-6c0d605e9054-kube-api-access-dhvw6\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.744690 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.744656    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-data\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.744946 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.744698    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4dc6f73-5287-4f71-b368-6c0d605e9054-istiod-ca-cert\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.744946 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.744794    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-credential-socket\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.745059 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.744988    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-workload-certs\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.745059 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.745035    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-workload-socket\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.745223 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.745199    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-data\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.745455 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.745439    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4dc6f73-5287-4f71-b368-6c0d605e9054-istiod-ca-cert\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.746814 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.746791    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-envoy\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.747038 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.747019    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-podinfo\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.800105 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.800069    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-token\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.804181 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.804146    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhvw6\" (UniqueName: \"kubernetes.io/projected/e4dc6f73-5287-4f71-b368-6c0d605e9054-kube-api-access-dhvw6\") pod \"e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") " pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:48.833075 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:48.833042    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:49.278122 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:49.278095    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"]
Oct 30 19:14:49.280886 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:14:49.280855    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4dc6f73_5287_4f71_b368_6c0d605e9054.slice/crio-1c43718f8ef73cb842a15c2cbaa0ca2a45d4141912477cbb88d747d0d078f06a WatchSource:0}: Error finding container 1c43718f8ef73cb842a15c2cbaa0ca2a45d4141912477cbb88d747d0d078f06a: Status 404 returned error can't find the container with id 1c43718f8ef73cb842a15c2cbaa0ca2a45d4141912477cbb88d747d0d078f06a
Oct 30 19:14:49.283842 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:49.283803    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:14:49.283927 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:49.283892    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:14:49.283966 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:49.283933    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:14:49.981364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:49.981272    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc" event={"ID":"e4dc6f73-5287-4f71-b368-6c0d605e9054","Type":"ContainerStarted","Data":"5eed08e9680e8ca3d075b8099176687bfbc007b0317d12f759a3a42fb37e5a4c"}
Oct 30 19:14:49.981364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:49.981342    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc" event={"ID":"e4dc6f73-5287-4f71-b368-6c0d605e9054","Type":"ContainerStarted","Data":"1c43718f8ef73cb842a15c2cbaa0ca2a45d4141912477cbb88d747d0d078f06a"}
Oct 30 19:14:50.099636 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:50.098164    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc" podStartSLOduration=2.098132204 podStartE2EDuration="2.098132204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:50.097526481 +0000 UTC m=+2287.623401580" watchObservedRunningTime="2026-10-30 19:14:50.098132204 +0000 UTC m=+2287.624007301"
Oct 30 19:14:50.833676 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:50.833639    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:50.835133 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:50.835105    4544 patch_prober.go:29] interesting pod/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 19:14:50.835260 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:50.835159    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc" podUID="e4dc6f73-5287-4f71-b368-6c0d605e9054" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 19:14:51.833754 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:51.833725    4544 patch_prober.go:29] interesting pod/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 19:14:51.834205 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:51.833773    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc" podUID="e4dc6f73-5287-4f71-b368-6c0d605e9054" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 19:14:52.834310 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:52.834273    4544 patch_prober.go:29] interesting pod/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 19:14:52.834794 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:52.834334    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc" podUID="e4dc6f73-5287-4f71-b368-6c0d605e9054" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 19:14:53.492169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:53.492106    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl" podUID="e2f8cdee-cc53-42b9-b70b-d025ef19bcd7" containerName="payload-pre-processing" containerID="cri-o://7d3ab6ccf0051e6b5d653906ed40f6e04079a4728008ae8ff860193f37bf4879" gracePeriod=25
Oct 30 19:14:53.834622 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:53.833883    4544 patch_prober.go:29] interesting pod/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 19:14:53.834622 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:53.833948    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc" podUID="e4dc6f73-5287-4f71-b368-6c0d605e9054" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 19:14:54.834067 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:54.834032    4544 patch_prober.go:29] interesting pod/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 19:14:54.834240 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:54.834092    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc" podUID="e4dc6f73-5287-4f71-b368-6c0d605e9054" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 19:14:55.833418 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:55.833382    4544 patch_prober.go:29] interesting pod/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 30 19:14:55.833809 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:55.833446    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc" podUID="e4dc6f73-5287-4f71-b368-6c0d605e9054" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 30 19:14:56.838226 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:56.838196    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:56.838595 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:56.838460    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:56.839238 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:56.839217    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:14:57.907927 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:14:57.907892    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl" podUID="e2f8cdee-cc53-42b9-b70b-d025ef19bcd7" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:00.234344 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.234314    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x"]
Oct 30 19:15:00.238725 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.238702    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr"]
Oct 30 19:15:00.238856 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.238828    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x"
Oct 30 19:15:00.251775 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.251744    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr"
Oct 30 19:15:00.319391 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.319353    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7vzt\" (UniqueName: \"kubernetes.io/projected/8403daa2-a502-4d25-aea1-0558f21d6e7b-kube-api-access-z7vzt\") pod \"maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr\" (UID: \"8403daa2-a502-4d25-aea1-0558f21d6e7b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr"
Oct 30 19:15:00.319569 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.319429    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fdbv\" (UniqueName: \"kubernetes.io/projected/6652a8f9-4c8c-43f2-be7f-9f8025690485-kube-api-access-5fdbv\") pod \"maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x\" (UID: \"6652a8f9-4c8c-43f2-be7f-9f8025690485\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x"
Oct 30 19:15:00.319656 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.319569    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6652a8f9-4c8c-43f2-be7f-9f8025690485-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x\" (UID: \"6652a8f9-4c8c-43f2-be7f-9f8025690485\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x"
Oct 30 19:15:00.319716 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.319674    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8403daa2-a502-4d25-aea1-0558f21d6e7b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr\" (UID: \"8403daa2-a502-4d25-aea1-0558f21d6e7b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr"
Oct 30 19:15:00.332221 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.332193    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-clcpt\""
Oct 30 19:15:00.360724 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.360696    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x"]
Oct 30 19:15:00.362921 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.362891    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr"]
Oct 30 19:15:00.420624 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.420574    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6652a8f9-4c8c-43f2-be7f-9f8025690485-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x\" (UID: \"6652a8f9-4c8c-43f2-be7f-9f8025690485\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x"
Oct 30 19:15:00.420770 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.420675    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8403daa2-a502-4d25-aea1-0558f21d6e7b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr\" (UID: \"8403daa2-a502-4d25-aea1-0558f21d6e7b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr"
Oct 30 19:15:00.420908 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.420890    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7vzt\" (UniqueName: \"kubernetes.io/projected/8403daa2-a502-4d25-aea1-0558f21d6e7b-kube-api-access-z7vzt\") pod \"maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr\" (UID: \"8403daa2-a502-4d25-aea1-0558f21d6e7b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr"
Oct 30 19:15:00.420977 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.420954    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fdbv\" (UniqueName: \"kubernetes.io/projected/6652a8f9-4c8c-43f2-be7f-9f8025690485-kube-api-access-5fdbv\") pod \"maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x\" (UID: \"6652a8f9-4c8c-43f2-be7f-9f8025690485\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x"
Oct 30 19:15:00.421308 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.421273    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6652a8f9-4c8c-43f2-be7f-9f8025690485-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x\" (UID: \"6652a8f9-4c8c-43f2-be7f-9f8025690485\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x"
Oct 30 19:15:00.421406 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.421331    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8403daa2-a502-4d25-aea1-0558f21d6e7b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr\" (UID: \"8403daa2-a502-4d25-aea1-0558f21d6e7b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr"
Oct 30 19:15:00.475202 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.475171    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7vzt\" (UniqueName: \"kubernetes.io/projected/8403daa2-a502-4d25-aea1-0558f21d6e7b-kube-api-access-z7vzt\") pod \"maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr\" (UID: \"8403daa2-a502-4d25-aea1-0558f21d6e7b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr"
Oct 30 19:15:00.475443 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.475421    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fdbv\" (UniqueName: \"kubernetes.io/projected/6652a8f9-4c8c-43f2-be7f-9f8025690485-kube-api-access-5fdbv\") pod \"maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x\" (UID: \"6652a8f9-4c8c-43f2-be7f-9f8025690485\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x"
Oct 30 19:15:00.567799 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.567772    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x"
Oct 30 19:15:00.573232 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.573209    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr"
Oct 30 19:15:00.945754 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.945726    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x"]
Oct 30 19:15:00.947334 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:15:00.947302    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6652a8f9_4c8c_43f2_be7f_9f8025690485.slice/crio-a1b0547021ae732fa41cd2c1035f758ffbe8287ef9cb7c81063fc7498ca9cad7 WatchSource:0}: Error finding container a1b0547021ae732fa41cd2c1035f758ffbe8287ef9cb7c81063fc7498ca9cad7: Status 404 returned error can't find the container with id a1b0547021ae732fa41cd2c1035f758ffbe8287ef9cb7c81063fc7498ca9cad7
Oct 30 19:15:00.995948 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:00.995920    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr"]
Oct 30 19:15:00.997214 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:15:00.997188    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8403daa2_a502_4d25_aea1_0558f21d6e7b.slice/crio-8285479bfe6ad93a6aea79b6d62778c2ea401e931224afde48e0bbb02fd1594f WatchSource:0}: Error finding container 8285479bfe6ad93a6aea79b6d62778c2ea401e931224afde48e0bbb02fd1594f: Status 404 returned error can't find the container with id 8285479bfe6ad93a6aea79b6d62778c2ea401e931224afde48e0bbb02fd1594f
Oct 30 19:15:01.079619 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:01.079570    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr" event={"ID":"8403daa2-a502-4d25-aea1-0558f21d6e7b","Type":"ContainerStarted","Data":"8285479bfe6ad93a6aea79b6d62778c2ea401e931224afde48e0bbb02fd1594f"}
Oct 30 19:15:01.080971 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:01.080944    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x" event={"ID":"6652a8f9-4c8c-43f2-be7f-9f8025690485","Type":"ContainerStarted","Data":"a1b0547021ae732fa41cd2c1035f758ffbe8287ef9cb7c81063fc7498ca9cad7"}
Oct 30 19:15:01.851486 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:01.851450    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"]
Oct 30 19:15:01.852038 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:01.851668    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c" podUID="75bf6071-0e48-4e7e-b1d3-284f2c14e52e" containerName="maas-api" containerID="cri-o://13c3de8698598028fb42a3eff8508dd7e2b301e4a3377cb05a85b9c80908e3d9" gracePeriod=30
Oct 30 19:15:02.115791 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:02.115691    4544 generic.go:356] "Generic (PLEG): container finished" podID="75bf6071-0e48-4e7e-b1d3-284f2c14e52e" containerID="13c3de8698598028fb42a3eff8508dd7e2b301e4a3377cb05a85b9c80908e3d9" exitCode=0
Oct 30 19:15:02.115791 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:02.115775    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c" event={"ID":"75bf6071-0e48-4e7e-b1d3-284f2c14e52e","Type":"ContainerDied","Data":"13c3de8698598028fb42a3eff8508dd7e2b301e4a3377cb05a85b9c80908e3d9"}
Oct 30 19:15:02.171344 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:02.171313    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:15:02.251115 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:02.251079    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-metrics-certs\") pod \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\" (UID: \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\") "
Oct 30 19:15:02.251339 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:02.251307    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-maas-api-tls\") pod \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\" (UID: \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\") "
Oct 30 19:15:02.261626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:02.251954    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-openshift-service-ca\") pod \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\" (UID: \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\") "
Oct 30 19:15:02.261626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:02.252490    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-openshift-service-ca" pod "75bf6071-0e48-4e7e-b1d3-284f2c14e52e" (UID: "75bf6071-0e48-4e7e-b1d3-284f2c14e52e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:02.261626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:02.253761    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-kube-api-access-8dpwq\" (UniqueName: \"kubernetes.io/projected/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-kube-api-access-8dpwq\") pod \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\" (UID: \"75bf6071-0e48-4e7e-b1d3-284f2c14e52e\") "
Oct 30 19:15:02.261626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:02.254472    4544 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-openshift-service-ca\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:02.261626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:02.258756    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-kube-api-access-8dpwq" pod "75bf6071-0e48-4e7e-b1d3-284f2c14e52e" (UID: "75bf6071-0e48-4e7e-b1d3-284f2c14e52e"). InnerVolumeSpecName "kube-api-access-8dpwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:02.276829 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:02.271686    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-metrics-certs" pod "75bf6071-0e48-4e7e-b1d3-284f2c14e52e" (UID: "75bf6071-0e48-4e7e-b1d3-284f2c14e52e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:02.280227 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:02.280191    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-maas-api-tls" pod "75bf6071-0e48-4e7e-b1d3-284f2c14e52e" (UID: "75bf6071-0e48-4e7e-b1d3-284f2c14e52e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:02.355967 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:02.355937    4544 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-maas-api-tls\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:02.355967 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:02.355965    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dpwq\" (UniqueName: \"kubernetes.io/projected/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-kube-api-access-8dpwq\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:02.355967 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:02.355977    4544 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75bf6071-0e48-4e7e-b1d3-284f2c14e52e-metrics-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:03.123330 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:03.123293    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c" event={"ID":"75bf6071-0e48-4e7e-b1d3-284f2c14e52e","Type":"ContainerDied","Data":"918b774b86ad27d2d711a0ab276edeb5c2e83a34a7f97d141e0ad43589a76449"}
Oct 30 19:15:03.123828 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:03.123341    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"
Oct 30 19:15:03.123828 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:03.123357    4544 scope.go:122] "RemoveContainer" containerID="13c3de8698598028fb42a3eff8508dd7e2b301e4a3377cb05a85b9c80908e3d9"
Oct 30 19:15:03.279712 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:03.279666    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"]
Oct 30 19:15:03.325668 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:03.325632    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-091e8f-7d9d69c66b-wgx6c"]
Oct 30 19:15:04.606583 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:04.606547    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75bf6071-0e48-4e7e-b1d3-284f2c14e52e" path="/var/lib/kubelet/pods/75bf6071-0e48-4e7e-b1d3-284f2c14e52e/volumes"
Oct 30 19:15:05.543145 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:15:05.543116    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:15:06.151015 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:06.150923    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr" event={"ID":"8403daa2-a502-4d25-aea1-0558f21d6e7b","Type":"ContainerStarted","Data":"b272087e3a0e58f4fc2f27a5719431b58f7337850d84a8f620d6f34c9c6a34ee"}
Oct 30 19:15:06.263819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:06.263755    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr" podStartSLOduration=1.77151948 podStartE2EDuration="6.263735269s" totalImagesPullingTime="4.492215789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:06.261896742 +0000 UTC m=+2303.787771863" watchObservedRunningTime="2026-10-30 19:15:06.263735269 +0000 UTC m=+2303.789610365"
Oct 30 19:15:07.171502 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:07.171463    4544 generic.go:356] "Generic (PLEG): container finished" podID="8403daa2-a502-4d25-aea1-0558f21d6e7b" containerID="b272087e3a0e58f4fc2f27a5719431b58f7337850d84a8f620d6f34c9c6a34ee" exitCode=60
Oct 30 19:15:07.171502 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:07.171494    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr" event={"ID":"8403daa2-a502-4d25-aea1-0558f21d6e7b","Type":"ContainerDied","Data":"b272087e3a0e58f4fc2f27a5719431b58f7337850d84a8f620d6f34c9c6a34ee"}
Oct 30 19:15:07.172086 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:07.171942    4544 scope.go:122] "RemoveContainer" containerID="b272087e3a0e58f4fc2f27a5719431b58f7337850d84a8f620d6f34c9c6a34ee"
Oct 30 19:15:07.178383 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:07.177818    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x" event={"ID":"6652a8f9-4c8c-43f2-be7f-9f8025690485","Type":"ContainerStarted","Data":"5811c26588cd9cc4f1a435dcfd1c57ed9e873d2a0376e292b8aba10389a59af5"}
Oct 30 19:15:07.391419 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:07.391355    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x" podStartSLOduration=1.905236046 podStartE2EDuration="7.391334647s" totalImagesPullingTime="5.486098601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:07.389641323 +0000 UTC m=+2304.915516413" watchObservedRunningTime="2026-10-30 19:15:07.391334647 +0000 UTC m=+2304.917209746"
Oct 30 19:15:07.907795 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:07.907760    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl" podUID="e2f8cdee-cc53-42b9-b70b-d025ef19bcd7" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:08.186597 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:08.186562    4544 generic.go:356] "Generic (PLEG): container finished" podID="6652a8f9-4c8c-43f2-be7f-9f8025690485" containerID="5811c26588cd9cc4f1a435dcfd1c57ed9e873d2a0376e292b8aba10389a59af5" exitCode=60
Oct 30 19:15:08.186985 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:08.186643    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x" event={"ID":"6652a8f9-4c8c-43f2-be7f-9f8025690485","Type":"ContainerDied","Data":"5811c26588cd9cc4f1a435dcfd1c57ed9e873d2a0376e292b8aba10389a59af5"}
Oct 30 19:15:08.186985 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:08.186951    4544 scope.go:122] "RemoveContainer" containerID="5811c26588cd9cc4f1a435dcfd1c57ed9e873d2a0376e292b8aba10389a59af5"
Oct 30 19:15:08.188979 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:08.188958    4544 generic.go:356] "Generic (PLEG): container finished" podID="8403daa2-a502-4d25-aea1-0558f21d6e7b" containerID="fb75eef44549fd32a77d4d49b674d3466febabcc6f9e750cd412cb736bbe7db1" exitCode=60
Oct 30 19:15:08.189091 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:08.189008    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr" event={"ID":"8403daa2-a502-4d25-aea1-0558f21d6e7b","Type":"ContainerDied","Data":"fb75eef44549fd32a77d4d49b674d3466febabcc6f9e750cd412cb736bbe7db1"}
Oct 30 19:15:08.189091 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:08.189048    4544 scope.go:122] "RemoveContainer" containerID="b272087e3a0e58f4fc2f27a5719431b58f7337850d84a8f620d6f34c9c6a34ee"
Oct 30 19:15:08.189365 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:08.189304    4544 scope.go:122] "RemoveContainer" containerID="fb75eef44549fd32a77d4d49b674d3466febabcc6f9e750cd412cb736bbe7db1"
Oct 30 19:15:08.189652 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:15:08.189587    4544 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr_odh-ai-gateway-infra(8403daa2-a502-4d25-aea1-0558f21d6e7b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr" podUID="8403daa2-a502-4d25-aea1-0558f21d6e7b"
Oct 30 19:15:09.199732 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:09.199707    4544 scope.go:122] "RemoveContainer" containerID="fb75eef44549fd32a77d4d49b674d3466febabcc6f9e750cd412cb736bbe7db1"
Oct 30 19:15:09.200109 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:15:09.200032    4544 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr_odh-ai-gateway-infra(8403daa2-a502-4d25-aea1-0558f21d6e7b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr" podUID="8403daa2-a502-4d25-aea1-0558f21d6e7b"
Oct 30 19:15:09.201280 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:09.201256    4544 generic.go:356] "Generic (PLEG): container finished" podID="6652a8f9-4c8c-43f2-be7f-9f8025690485" containerID="f7c487ab04e5d847f018385ded0bc1e24c05dc4e3903e406e784420e9ff78382" exitCode=60
Oct 30 19:15:09.201338 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:09.201299    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x" event={"ID":"6652a8f9-4c8c-43f2-be7f-9f8025690485","Type":"ContainerDied","Data":"f7c487ab04e5d847f018385ded0bc1e24c05dc4e3903e406e784420e9ff78382"}
Oct 30 19:15:09.201386 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:09.201349    4544 scope.go:122] "RemoveContainer" containerID="5811c26588cd9cc4f1a435dcfd1c57ed9e873d2a0376e292b8aba10389a59af5"
Oct 30 19:15:09.201612 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:09.201577    4544 scope.go:122] "RemoveContainer" containerID="f7c487ab04e5d847f018385ded0bc1e24c05dc4e3903e406e784420e9ff78382"
Oct 30 19:15:09.201893 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:15:09.201867    4544 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x_odh-ai-gateway-infra(6652a8f9-4c8c-43f2-be7f-9f8025690485)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x" podUID="6652a8f9-4c8c-43f2-be7f-9f8025690485"
Oct 30 19:15:10.209980 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:10.209954    4544 scope.go:122] "RemoveContainer" containerID="f7c487ab04e5d847f018385ded0bc1e24c05dc4e3903e406e784420e9ff78382"
Oct 30 19:15:10.210365 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:15:10.210233    4544 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x_odh-ai-gateway-infra(6652a8f9-4c8c-43f2-be7f-9f8025690485)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x" podUID="6652a8f9-4c8c-43f2-be7f-9f8025690485"
Oct 30 19:15:12.041320 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:12.041283    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"]
Oct 30 19:15:12.041860 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:12.041502    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb" podUID="b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" containerName="istio-proxy" containerID="cri-o://c9a86d092f980a55d0751a71348126aa7dfc07fee891106feb3f137a7c14700e" gracePeriod=30
Oct 30 19:15:13.753213 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:13.753142    4544 patch_prober.go:29] interesting pod/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:15:13.753645 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:13.753202    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb" podUID="b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:15:13.761514 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:13.761491    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl"
Oct 30 19:15:13.803196 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:13.803165    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2f8cdee-cc53-42b9-b70b-d025ef19bcd7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2f8cdee-cc53-42b9-b70b-d025ef19bcd7-plugins-config-volume\") pod \"e2f8cdee-cc53-42b9-b70b-d025ef19bcd7\" (UID: \"e2f8cdee-cc53-42b9-b70b-d025ef19bcd7\") "
Oct 30 19:15:13.803359 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:13.803311    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2f8cdee-cc53-42b9-b70b-d025ef19bcd7-kube-api-access-8kmpc\" (UniqueName: \"kubernetes.io/projected/e2f8cdee-cc53-42b9-b70b-d025ef19bcd7-kube-api-access-8kmpc\") pod \"e2f8cdee-cc53-42b9-b70b-d025ef19bcd7\" (UID: \"e2f8cdee-cc53-42b9-b70b-d025ef19bcd7\") "
Oct 30 19:15:13.803644 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:13.803594    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2f8cdee-cc53-42b9-b70b-d025ef19bcd7-plugins-config-volume" pod "e2f8cdee-cc53-42b9-b70b-d025ef19bcd7" (UID: "e2f8cdee-cc53-42b9-b70b-d025ef19bcd7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:13.803750 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:13.803734    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2f8cdee-cc53-42b9-b70b-d025ef19bcd7-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.805942 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:13.805914    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2f8cdee-cc53-42b9-b70b-d025ef19bcd7-kube-api-access-8kmpc" pod "e2f8cdee-cc53-42b9-b70b-d025ef19bcd7" (UID: "e2f8cdee-cc53-42b9-b70b-d025ef19bcd7"). InnerVolumeSpecName "kube-api-access-8kmpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:13.905581 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:13.905544    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kmpc\" (UniqueName: \"kubernetes.io/projected/e2f8cdee-cc53-42b9-b70b-d025ef19bcd7-kube-api-access-8kmpc\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:14.235826 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:14.235787    4544 generic.go:356] "Generic (PLEG): container finished" podID="e2f8cdee-cc53-42b9-b70b-d025ef19bcd7" containerID="7d3ab6ccf0051e6b5d653906ed40f6e04079a4728008ae8ff860193f37bf4879" exitCode=0
Oct 30 19:15:14.235999 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:14.235837    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl"
Oct 30 19:15:14.235999 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:14.235889    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl" event={"ID":"e2f8cdee-cc53-42b9-b70b-d025ef19bcd7","Type":"ContainerDied","Data":"7d3ab6ccf0051e6b5d653906ed40f6e04079a4728008ae8ff860193f37bf4879"}
Oct 30 19:15:14.235999 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:14.235949    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl" event={"ID":"e2f8cdee-cc53-42b9-b70b-d025ef19bcd7","Type":"ContainerDied","Data":"9e201d8c86a18c99d9372e238b75169b3320994eef47fcbc1ce366f24a82a667"}
Oct 30 19:15:14.235999 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:14.235979    4544 scope.go:122] "RemoveContainer" containerID="7d3ab6ccf0051e6b5d653906ed40f6e04079a4728008ae8ff860193f37bf4879"
Oct 30 19:15:14.246320 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:14.246297    4544 scope.go:122] "RemoveContainer" containerID="7d3ab6ccf0051e6b5d653906ed40f6e04079a4728008ae8ff860193f37bf4879"
Oct 30 19:15:14.246585 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:15:14.246564    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d3ab6ccf0051e6b5d653906ed40f6e04079a4728008ae8ff860193f37bf4879\": container with ID starting with 7d3ab6ccf0051e6b5d653906ed40f6e04079a4728008ae8ff860193f37bf4879 not found: ID does not exist" containerID="7d3ab6ccf0051e6b5d653906ed40f6e04079a4728008ae8ff860193f37bf4879"
Oct 30 19:15:14.246680 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:14.246592    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d3ab6ccf0051e6b5d653906ed40f6e04079a4728008ae8ff860193f37bf4879"} err="failed to get container status \"7d3ab6ccf0051e6b5d653906ed40f6e04079a4728008ae8ff860193f37bf4879\": rpc error: code = NotFound desc = could not find container \"7d3ab6ccf0051e6b5d653906ed40f6e04079a4728008ae8ff860193f37bf4879\": container with ID starting with 7d3ab6ccf0051e6b5d653906ed40f6e04079a4728008ae8ff860193f37bf4879 not found: ID does not exist"
Oct 30 19:15:14.391959 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:14.391922    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl"]
Oct 30 19:15:14.440202 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:14.440166    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-649d16-7449498687-dtzpl"]
Oct 30 19:15:14.602419 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:14.602388    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e2f8cdee-cc53-42b9-b70b-d025ef19bcd7" path="/var/lib/kubelet/pods/e2f8cdee-cc53-42b9-b70b-d025ef19bcd7/volumes"
Oct 30 19:15:17.291162 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.291126    4544 generic.go:356] "Generic (PLEG): container finished" podID="b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" containerID="c9a86d092f980a55d0751a71348126aa7dfc07fee891106feb3f137a7c14700e" exitCode=0
Oct 30 19:15:17.291841 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.291815    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb" event={"ID":"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e","Type":"ContainerDied","Data":"c9a86d092f980a55d0751a71348126aa7dfc07fee891106feb3f137a7c14700e"}
Oct 30 19:15:17.294054 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.293862    4544 scope.go:122] "RemoveContainer" containerID="89c0a630c5e2f5178593997064c91bf04429f044c020b803cb6ef3bb4c3ae237"
Oct 30 19:15:17.363084 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.362673    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:15:17.450760 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.450731    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-workload-certs\") pod \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") "
Oct 30 19:15:17.450929 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.450788    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-workload-socket\") pod \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") "
Oct 30 19:15:17.450929 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.450863    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-envoy\") pod \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") "
Oct 30 19:15:17.450929 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.450893    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-credential-socket\") pod \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") "
Oct 30 19:15:17.450929 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.450922    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-kube-api-access-cdjss\" (UniqueName: \"kubernetes.io/projected/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-kube-api-access-cdjss\") pod \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") "
Oct 30 19:15:17.451176 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.451003    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-data\") pod \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") "
Oct 30 19:15:17.451176 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.451019    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-workload-certs" pod "b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" (UID: "b3c4f1eb-a61c-4184-8ae6-afc8a54e024e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:17.451176 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.451072    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istiod-ca-cert\") pod \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") "
Oct 30 19:15:17.451176 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.451089    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-workload-socket" pod "b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" (UID: "b3c4f1eb-a61c-4184-8ae6-afc8a54e024e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:17.451176 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.451105    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-podinfo\") pod \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") "
Oct 30 19:15:17.451435 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.451174    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-credential-socket" pod "b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" (UID: "b3c4f1eb-a61c-4184-8ae6-afc8a54e024e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:17.451435 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.451224    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-token\" (UniqueName: \"kubernetes.io/projected/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-token\") pod \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\" (UID: \"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e\") "
Oct 30 19:15:17.451555 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.451500    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-data" pod "b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" (UID: "b3c4f1eb-a61c-4184-8ae6-afc8a54e024e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:17.451718 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.451693    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istiod-ca-cert" pod "b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" (UID: "b3c4f1eb-a61c-4184-8ae6-afc8a54e024e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:17.451812 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.451787    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:17.451812 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.451808    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:17.451925 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.451823    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:17.451925 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.451838    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:17.453313 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.453289    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-token" pod "b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" (UID: "b3c4f1eb-a61c-4184-8ae6-afc8a54e024e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:17.453398 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.453336    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-podinfo" pod "b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" (UID: "b3c4f1eb-a61c-4184-8ae6-afc8a54e024e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:15:17.453540 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.453514    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-envoy" pod "b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" (UID: "b3c4f1eb-a61c-4184-8ae6-afc8a54e024e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:17.453656 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.453572    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-kube-api-access-cdjss" pod "b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" (UID: "b3c4f1eb-a61c-4184-8ae6-afc8a54e024e"). InnerVolumeSpecName "kube-api-access-cdjss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:17.530740 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.530708    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"]
Oct 30 19:15:17.553285 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.553250    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:17.553285 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.553282    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:17.553483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.553295    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:17.553483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.553308    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:17.553483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:17.553320    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdjss\" (UniqueName: \"kubernetes.io/projected/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e-kube-api-access-cdjss\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:18.299668 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:18.299635    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb" event={"ID":"b3c4f1eb-a61c-4184-8ae6-afc8a54e024e","Type":"ContainerDied","Data":"e4b6758a9ca5006027e7bf512b385891eaab903288db8bfa11f1cc2c8e65c076"}
Oct 30 19:15:18.300063 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:18.299670    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"
Oct 30 19:15:18.300063 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:18.299696    4544 scope.go:122] "RemoveContainer" containerID="c9a86d092f980a55d0751a71348126aa7dfc07fee891106feb3f137a7c14700e"
Oct 30 19:15:18.458043 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:18.458004    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"]
Oct 30 19:15:18.504637 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:18.504567    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-091e8f-openshift-default-7b97649989-87qpb"]
Oct 30 19:15:18.603691 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:18.603622    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" path="/var/lib/kubelet/pods/b3c4f1eb-a61c-4184-8ae6-afc8a54e024e/volumes"
Oct 30 19:15:19.595801 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:19.595771    4544 scope.go:122] "RemoveContainer" containerID="f7c487ab04e5d847f018385ded0bc1e24c05dc4e3903e406e784420e9ff78382"
Oct 30 19:15:20.315062 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:20.315025    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x" event={"ID":"6652a8f9-4c8c-43f2-be7f-9f8025690485","Type":"ContainerStarted","Data":"2f5f38a353f286952e9414ef2ebcae536aeefa7341c9b3cb5efe885547b5a8c3"}
Oct 30 19:15:21.067438 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:21.067404    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"]
Oct 30 19:15:21.067906 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:21.067644    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2" podUID="fe43caaf-eb11-4b75-a185-66ab33940e27" containerName="istio-proxy" containerID="cri-o://4f39785031d4b8611956b238a42fc8ec7a06862568d6a88191ca837cdfc1333b" gracePeriod=30
Oct 30 19:15:21.322659 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:21.322555    4544 generic.go:356] "Generic (PLEG): container finished" podID="6652a8f9-4c8c-43f2-be7f-9f8025690485" containerID="2f5f38a353f286952e9414ef2ebcae536aeefa7341c9b3cb5efe885547b5a8c3" exitCode=60
Oct 30 19:15:21.322659 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:21.322635    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x" event={"ID":"6652a8f9-4c8c-43f2-be7f-9f8025690485","Type":"ContainerDied","Data":"2f5f38a353f286952e9414ef2ebcae536aeefa7341c9b3cb5efe885547b5a8c3"}
Oct 30 19:15:21.322877 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:21.322694    4544 scope.go:122] "RemoveContainer" containerID="f7c487ab04e5d847f018385ded0bc1e24c05dc4e3903e406e784420e9ff78382"
Oct 30 19:15:21.323056 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:21.323039    4544 scope.go:122] "RemoveContainer" containerID="2f5f38a353f286952e9414ef2ebcae536aeefa7341c9b3cb5efe885547b5a8c3"
Oct 30 19:15:21.323323 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:15:21.323297    4544 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x_odh-ai-gateway-infra(6652a8f9-4c8c-43f2-be7f-9f8025690485)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x" podUID="6652a8f9-4c8c-43f2-be7f-9f8025690485"
Oct 30 19:15:21.442057 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:21.442025    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x"]
Oct 30 19:15:21.604368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:21.604269    4544 scope.go:122] "RemoveContainer" containerID="fb75eef44549fd32a77d4d49b674d3466febabcc6f9e750cd412cb736bbe7db1"
Oct 30 19:15:22.331042 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:22.331004    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr" event={"ID":"8403daa2-a502-4d25-aea1-0558f21d6e7b","Type":"ContainerStarted","Data":"c51059d58cb7c7ee31c1a948ae24cbe24de6b408505539db7a7af6bb4afcfe97"}
Oct 30 19:15:22.507119 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:22.507087    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x"
Oct 30 19:15:22.639042 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:22.638956    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6652a8f9-4c8c-43f2-be7f-9f8025690485-kube-api-access-5fdbv\" (UniqueName: \"kubernetes.io/projected/6652a8f9-4c8c-43f2-be7f-9f8025690485-kube-api-access-5fdbv\") pod \"6652a8f9-4c8c-43f2-be7f-9f8025690485\" (UID: \"6652a8f9-4c8c-43f2-be7f-9f8025690485\") "
Oct 30 19:15:22.639213 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:22.639149    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6652a8f9-4c8c-43f2-be7f-9f8025690485-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6652a8f9-4c8c-43f2-be7f-9f8025690485-maas-api-service-ca\") pod \"6652a8f9-4c8c-43f2-be7f-9f8025690485\" (UID: \"6652a8f9-4c8c-43f2-be7f-9f8025690485\") "
Oct 30 19:15:22.639511 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:22.639484    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6652a8f9-4c8c-43f2-be7f-9f8025690485-maas-api-service-ca" pod "6652a8f9-4c8c-43f2-be7f-9f8025690485" (UID: "6652a8f9-4c8c-43f2-be7f-9f8025690485"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:22.639698 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:22.639592    4544 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6652a8f9-4c8c-43f2-be7f-9f8025690485-maas-api-service-ca\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:22.641257 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:22.641233    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6652a8f9-4c8c-43f2-be7f-9f8025690485-kube-api-access-5fdbv" pod "6652a8f9-4c8c-43f2-be7f-9f8025690485" (UID: "6652a8f9-4c8c-43f2-be7f-9f8025690485"). InnerVolumeSpecName "kube-api-access-5fdbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:22.740443 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:22.740409    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fdbv\" (UniqueName: \"kubernetes.io/projected/6652a8f9-4c8c-43f2-be7f-9f8025690485-kube-api-access-5fdbv\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:23.341446 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:23.341410    4544 generic.go:356] "Generic (PLEG): container finished" podID="8403daa2-a502-4d25-aea1-0558f21d6e7b" containerID="c51059d58cb7c7ee31c1a948ae24cbe24de6b408505539db7a7af6bb4afcfe97" exitCode=60
Oct 30 19:15:23.341914 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:23.341481    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr" event={"ID":"8403daa2-a502-4d25-aea1-0558f21d6e7b","Type":"ContainerDied","Data":"c51059d58cb7c7ee31c1a948ae24cbe24de6b408505539db7a7af6bb4afcfe97"}
Oct 30 19:15:23.341914 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:23.341562    4544 scope.go:122] "RemoveContainer" containerID="fb75eef44549fd32a77d4d49b674d3466febabcc6f9e750cd412cb736bbe7db1"
Oct 30 19:15:23.342041 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:23.341970    4544 scope.go:122] "RemoveContainer" containerID="c51059d58cb7c7ee31c1a948ae24cbe24de6b408505539db7a7af6bb4afcfe97"
Oct 30 19:15:23.342291 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:15:23.342269    4544 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr_odh-ai-gateway-infra(8403daa2-a502-4d25-aea1-0558f21d6e7b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr" podUID="8403daa2-a502-4d25-aea1-0558f21d6e7b"
Oct 30 19:15:23.343153 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:23.343127    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x" event={"ID":"6652a8f9-4c8c-43f2-be7f-9f8025690485","Type":"ContainerDied","Data":"a1b0547021ae732fa41cd2c1035f758ffbe8287ef9cb7c81063fc7498ca9cad7"}
Oct 30 19:15:23.343239 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:23.343137    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x"
Oct 30 19:15:23.365923 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:23.365901    4544 scope.go:122] "RemoveContainer" containerID="2f5f38a353f286952e9414ef2ebcae536aeefa7341c9b3cb5efe885547b5a8c3"
Oct 30 19:15:23.461696 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:23.461659    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr"]
Oct 30 19:15:23.554203 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:23.554171    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x"]
Oct 30 19:15:23.558808 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:23.558781    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2ef49a-29889795-tjk7x"]
Oct 30 19:15:24.507376 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:24.507346    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr"
Oct 30 19:15:24.613549 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:24.613458    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6652a8f9-4c8c-43f2-be7f-9f8025690485" path="/var/lib/kubelet/pods/6652a8f9-4c8c-43f2-be7f-9f8025690485/volumes"
Oct 30 19:15:24.663495 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:24.663462    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8403daa2-a502-4d25-aea1-0558f21d6e7b-kube-api-access-z7vzt\" (UniqueName: \"kubernetes.io/projected/8403daa2-a502-4d25-aea1-0558f21d6e7b-kube-api-access-z7vzt\") pod \"8403daa2-a502-4d25-aea1-0558f21d6e7b\" (UID: \"8403daa2-a502-4d25-aea1-0558f21d6e7b\") "
Oct 30 19:15:24.663694 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:24.663648    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8403daa2-a502-4d25-aea1-0558f21d6e7b-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8403daa2-a502-4d25-aea1-0558f21d6e7b-maas-api-service-ca\") pod \"8403daa2-a502-4d25-aea1-0558f21d6e7b\" (UID: \"8403daa2-a502-4d25-aea1-0558f21d6e7b\") "
Oct 30 19:15:24.664067 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:24.664035    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8403daa2-a502-4d25-aea1-0558f21d6e7b-maas-api-service-ca" pod "8403daa2-a502-4d25-aea1-0558f21d6e7b" (UID: "8403daa2-a502-4d25-aea1-0558f21d6e7b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:24.665587 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:24.665557    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8403daa2-a502-4d25-aea1-0558f21d6e7b-kube-api-access-z7vzt" pod "8403daa2-a502-4d25-aea1-0558f21d6e7b" (UID: "8403daa2-a502-4d25-aea1-0558f21d6e7b"). InnerVolumeSpecName "kube-api-access-z7vzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:24.765298 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:24.765265    4544 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8403daa2-a502-4d25-aea1-0558f21d6e7b-maas-api-service-ca\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:24.765298 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:24.765292    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7vzt\" (UniqueName: \"kubernetes.io/projected/8403daa2-a502-4d25-aea1-0558f21d6e7b-kube-api-access-z7vzt\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:25.138568 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:25.138534    4544 patch_prober.go:29] interesting pod/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:15:25.138749 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:25.138590    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2" podUID="fe43caaf-eb11-4b75-a185-66ab33940e27" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:15:25.358309 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:25.358281    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr"
Oct 30 19:15:25.358309 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:25.358300    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr" event={"ID":"8403daa2-a502-4d25-aea1-0558f21d6e7b","Type":"ContainerDied","Data":"8285479bfe6ad93a6aea79b6d62778c2ea401e931224afde48e0bbb02fd1594f"}
Oct 30 19:15:25.358582 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:25.358377    4544 scope.go:122] "RemoveContainer" containerID="c51059d58cb7c7ee31c1a948ae24cbe24de6b408505539db7a7af6bb4afcfe97"
Oct 30 19:15:25.521023 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:25.520994    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr"]
Oct 30 19:15:25.524225 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:25.524202    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f3a0e7-29889795-68lfr"]
Oct 30 19:15:26.340002 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.339944    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:15:26.369061 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.369032    4544 generic.go:356] "Generic (PLEG): container finished" podID="fe43caaf-eb11-4b75-a185-66ab33940e27" containerID="4f39785031d4b8611956b238a42fc8ec7a06862568d6a88191ca837cdfc1333b" exitCode=0
Oct 30 19:15:26.369226 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.369094    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"
Oct 30 19:15:26.369226 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.369128    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2" event={"ID":"fe43caaf-eb11-4b75-a185-66ab33940e27","Type":"ContainerDied","Data":"4f39785031d4b8611956b238a42fc8ec7a06862568d6a88191ca837cdfc1333b"}
Oct 30 19:15:26.369226 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.369178    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2" event={"ID":"fe43caaf-eb11-4b75-a185-66ab33940e27","Type":"ContainerDied","Data":"bb28c8b8c5fe6961b306a2666fdf58242508a30fef9baab3716e3879c00c68c5"}
Oct 30 19:15:26.369226 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.369218    4544 scope.go:122] "RemoveContainer" containerID="4f39785031d4b8611956b238a42fc8ec7a06862568d6a88191ca837cdfc1333b"
Oct 30 19:15:26.381222 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.381199    4544 scope.go:122] "RemoveContainer" containerID="46891d3a4bed07102f87a20dde1eab6e80588d4a99be77e94bcd8b1fdc6cf09e"
Oct 30 19:15:26.390471 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.390441    4544 scope.go:122] "RemoveContainer" containerID="4f39785031d4b8611956b238a42fc8ec7a06862568d6a88191ca837cdfc1333b"
Oct 30 19:15:26.390876 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:15:26.390855    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f39785031d4b8611956b238a42fc8ec7a06862568d6a88191ca837cdfc1333b\": container with ID starting with 4f39785031d4b8611956b238a42fc8ec7a06862568d6a88191ca837cdfc1333b not found: ID does not exist" containerID="4f39785031d4b8611956b238a42fc8ec7a06862568d6a88191ca837cdfc1333b"
Oct 30 19:15:26.390980 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.390884    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f39785031d4b8611956b238a42fc8ec7a06862568d6a88191ca837cdfc1333b"} err="failed to get container status \"4f39785031d4b8611956b238a42fc8ec7a06862568d6a88191ca837cdfc1333b\": rpc error: code = NotFound desc = could not find container \"4f39785031d4b8611956b238a42fc8ec7a06862568d6a88191ca837cdfc1333b\": container with ID starting with 4f39785031d4b8611956b238a42fc8ec7a06862568d6a88191ca837cdfc1333b not found: ID does not exist"
Oct 30 19:15:26.390980 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.390900    4544 scope.go:122] "RemoveContainer" containerID="46891d3a4bed07102f87a20dde1eab6e80588d4a99be77e94bcd8b1fdc6cf09e"
Oct 30 19:15:26.391179 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:15:26.391159    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46891d3a4bed07102f87a20dde1eab6e80588d4a99be77e94bcd8b1fdc6cf09e\": container with ID starting with 46891d3a4bed07102f87a20dde1eab6e80588d4a99be77e94bcd8b1fdc6cf09e not found: ID does not exist" containerID="46891d3a4bed07102f87a20dde1eab6e80588d4a99be77e94bcd8b1fdc6cf09e"
Oct 30 19:15:26.391236 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.391185    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46891d3a4bed07102f87a20dde1eab6e80588d4a99be77e94bcd8b1fdc6cf09e"} err="failed to get container status \"46891d3a4bed07102f87a20dde1eab6e80588d4a99be77e94bcd8b1fdc6cf09e\": rpc error: code = NotFound desc = could not find container \"46891d3a4bed07102f87a20dde1eab6e80588d4a99be77e94bcd8b1fdc6cf09e\": container with ID starting with 46891d3a4bed07102f87a20dde1eab6e80588d4a99be77e94bcd8b1fdc6cf09e not found: ID does not exist"
Oct 30 19:15:26.487305 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.487275    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe43caaf-eb11-4b75-a185-66ab33940e27-kube-api-access-klts9\" (UniqueName: \"kubernetes.io/projected/fe43caaf-eb11-4b75-a185-66ab33940e27-kube-api-access-klts9\") pod \"fe43caaf-eb11-4b75-a185-66ab33940e27\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") "
Oct 30 19:15:26.487503 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.487345    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-token\" (UniqueName: \"kubernetes.io/projected/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-token\") pod \"fe43caaf-eb11-4b75-a185-66ab33940e27\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") "
Oct 30 19:15:26.487503 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.487406    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-data\") pod \"fe43caaf-eb11-4b75-a185-66ab33940e27\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") "
Oct 30 19:15:26.487503 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.487473    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-workload-socket\") pod \"fe43caaf-eb11-4b75-a185-66ab33940e27\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") "
Oct 30 19:15:26.487737 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.487538    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-workload-certs\") pod \"fe43caaf-eb11-4b75-a185-66ab33940e27\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") "
Oct 30 19:15:26.487737 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.487567    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-envoy\") pod \"fe43caaf-eb11-4b75-a185-66ab33940e27\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") "
Oct 30 19:15:26.487737 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.487655    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-credential-socket\") pod \"fe43caaf-eb11-4b75-a185-66ab33940e27\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") "
Oct 30 19:15:26.487904 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.487771    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe43caaf-eb11-4b75-a185-66ab33940e27-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe43caaf-eb11-4b75-a185-66ab33940e27-istiod-ca-cert\") pod \"fe43caaf-eb11-4b75-a185-66ab33940e27\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") "
Oct 30 19:15:26.487904 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.487800    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-podinfo\") pod \"fe43caaf-eb11-4b75-a185-66ab33940e27\" (UID: \"fe43caaf-eb11-4b75-a185-66ab33940e27\") "
Oct 30 19:15:26.488011 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.487979    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-workload-socket" pod "fe43caaf-eb11-4b75-a185-66ab33940e27" (UID: "fe43caaf-eb11-4b75-a185-66ab33940e27"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:26.488071 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.488029    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-data" pod "fe43caaf-eb11-4b75-a185-66ab33940e27" (UID: "fe43caaf-eb11-4b75-a185-66ab33940e27"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:26.488472 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.488246    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-credential-socket" pod "fe43caaf-eb11-4b75-a185-66ab33940e27" (UID: "fe43caaf-eb11-4b75-a185-66ab33940e27"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:26.488472 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.488320    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe43caaf-eb11-4b75-a185-66ab33940e27-istiod-ca-cert" pod "fe43caaf-eb11-4b75-a185-66ab33940e27" (UID: "fe43caaf-eb11-4b75-a185-66ab33940e27"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:26.488472 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.488411    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:26.488472 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.488432    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:26.488472 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.488447    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:26.489012 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.488963    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-workload-certs" pod "fe43caaf-eb11-4b75-a185-66ab33940e27" (UID: "fe43caaf-eb11-4b75-a185-66ab33940e27"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:26.490175 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.490152    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-podinfo" pod "fe43caaf-eb11-4b75-a185-66ab33940e27" (UID: "fe43caaf-eb11-4b75-a185-66ab33940e27"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:15:26.490500 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.490480    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-token" pod "fe43caaf-eb11-4b75-a185-66ab33940e27" (UID: "fe43caaf-eb11-4b75-a185-66ab33940e27"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:26.490571 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.490517    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe43caaf-eb11-4b75-a185-66ab33940e27-kube-api-access-klts9" pod "fe43caaf-eb11-4b75-a185-66ab33940e27" (UID: "fe43caaf-eb11-4b75-a185-66ab33940e27"). InnerVolumeSpecName "kube-api-access-klts9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:26.491014 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.490993    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-envoy" pod "fe43caaf-eb11-4b75-a185-66ab33940e27" (UID: "fe43caaf-eb11-4b75-a185-66ab33940e27"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:26.593968 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.593632    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:26.593968 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.593670    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:26.593968 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.593685    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:26.593968 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.593699    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe43caaf-eb11-4b75-a185-66ab33940e27-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:26.593968 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.593713    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe43caaf-eb11-4b75-a185-66ab33940e27-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:26.593968 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.593729    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klts9\" (UniqueName: \"kubernetes.io/projected/fe43caaf-eb11-4b75-a185-66ab33940e27-kube-api-access-klts9\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:26.613465 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.613419    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8403daa2-a502-4d25-aea1-0558f21d6e7b" path="/var/lib/kubelet/pods/8403daa2-a502-4d25-aea1-0558f21d6e7b/volumes"
Oct 30 19:15:26.785445 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.785406    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"]
Oct 30 19:15:26.790582 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:26.790545    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-05565d-openshift-default-9c499798c-dmvl2"]
Oct 30 19:15:27.510069 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510036    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"]
Oct 30 19:15:27.510522 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510509    4544 cpu_manager.go:414] "containerMap: removing container" podUID="b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" containerName="istio-proxy"
Oct 30 19:15:27.510582 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510522    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" containerName="istio-proxy"
Oct 30 19:15:27.510582 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510536    4544 cpu_manager.go:414] "containerMap: removing container" podUID="8403daa2-a502-4d25-aea1-0558f21d6e7b" containerName="cleanup"
Oct 30 19:15:27.510582 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510541    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8403daa2-a502-4d25-aea1-0558f21d6e7b" containerName="cleanup"
Oct 30 19:15:27.510582 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510548    4544 cpu_manager.go:414] "containerMap: removing container" podUID="8403daa2-a502-4d25-aea1-0558f21d6e7b" containerName="cleanup"
Oct 30 19:15:27.510582 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510553    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8403daa2-a502-4d25-aea1-0558f21d6e7b" containerName="cleanup"
Oct 30 19:15:27.510582 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510563    4544 cpu_manager.go:414] "containerMap: removing container" podUID="6652a8f9-4c8c-43f2-be7f-9f8025690485" containerName="cleanup"
Oct 30 19:15:27.510582 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510569    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6652a8f9-4c8c-43f2-be7f-9f8025690485" containerName="cleanup"
Oct 30 19:15:27.510582 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510574    4544 cpu_manager.go:414] "containerMap: removing container" podUID="b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" containerName="istio-proxy"
Oct 30 19:15:27.510582 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510578    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" containerName="istio-proxy"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510586    4544 cpu_manager.go:414] "containerMap: removing container" podUID="e2f8cdee-cc53-42b9-b70b-d025ef19bcd7" containerName="payload-pre-processing"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510592    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2f8cdee-cc53-42b9-b70b-d025ef19bcd7" containerName="payload-pre-processing"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510625    4544 cpu_manager.go:414] "containerMap: removing container" podUID="fe43caaf-eb11-4b75-a185-66ab33940e27" containerName="istio-proxy"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510632    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe43caaf-eb11-4b75-a185-66ab33940e27" containerName="istio-proxy"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510643    4544 cpu_manager.go:414] "containerMap: removing container" podUID="75bf6071-0e48-4e7e-b1d3-284f2c14e52e" containerName="maas-api"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510650    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75bf6071-0e48-4e7e-b1d3-284f2c14e52e" containerName="maas-api"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510659    4544 cpu_manager.go:414] "containerMap: removing container" podUID="6652a8f9-4c8c-43f2-be7f-9f8025690485" containerName="cleanup"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510666    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6652a8f9-4c8c-43f2-be7f-9f8025690485" containerName="cleanup"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510683    4544 cpu_manager.go:414] "containerMap: removing container" podUID="fe43caaf-eb11-4b75-a185-66ab33940e27" containerName="istio-proxy"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510691    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe43caaf-eb11-4b75-a185-66ab33940e27" containerName="istio-proxy"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510814    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe43caaf-eb11-4b75-a185-66ab33940e27" containerName="istio-proxy"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510823    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="75bf6071-0e48-4e7e-b1d3-284f2c14e52e" containerName="maas-api"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510833    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="8403daa2-a502-4d25-aea1-0558f21d6e7b" containerName="cleanup"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510838    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="6652a8f9-4c8c-43f2-be7f-9f8025690485" containerName="cleanup"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510846    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe43caaf-eb11-4b75-a185-66ab33940e27" containerName="istio-proxy"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510854    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" containerName="istio-proxy"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510860    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="8403daa2-a502-4d25-aea1-0558f21d6e7b" containerName="cleanup"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510866    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="8403daa2-a502-4d25-aea1-0558f21d6e7b" containerName="cleanup"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510871    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="6652a8f9-4c8c-43f2-be7f-9f8025690485" containerName="cleanup"
Oct 30 19:15:27.510910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.510876    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2f8cdee-cc53-42b9-b70b-d025ef19bcd7" containerName="payload-pre-processing"
Oct 30 19:15:27.515737 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.515718    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.598895 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.598864    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"]
Oct 30 19:15:27.612232 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.612196    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-envoy\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.612512 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.612490    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-workload-socket\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.612726 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.612709    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-workload-certs\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.612841 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.612826    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-752cf\" (UniqueName: \"kubernetes.io/projected/90685ad5-9312-489e-ab58-84c12ebcc5ab-kube-api-access-752cf\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.612939 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.612926    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-token\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.613033 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.613021    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-credential-socket\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.613138 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.613125    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-data\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.613276 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.613261    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-podinfo\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.613410 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.613395    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90685ad5-9312-489e-ab58-84c12ebcc5ab-istiod-ca-cert\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.666784 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.663635    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-29de7b-openshift-default-dockercfg-4vm5m\""
Oct 30 19:15:27.714506 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.714453    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-podinfo\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.714727 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.714516    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90685ad5-9312-489e-ab58-84c12ebcc5ab-istiod-ca-cert\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.714727 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.714632    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-envoy\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.714727 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.714710    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-workload-socket\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.714899 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.714833    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-workload-certs\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.714899 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.714876    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-752cf\" (UniqueName: \"kubernetes.io/projected/90685ad5-9312-489e-ab58-84c12ebcc5ab-kube-api-access-752cf\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.715005 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.714903    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-token\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.715005 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.714929    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-credential-socket\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.715005 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.714960    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-data\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.715584 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.715325    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-workload-certs\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.715584 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.715344    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90685ad5-9312-489e-ab58-84c12ebcc5ab-istiod-ca-cert\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.715584 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.715547    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-workload-socket\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.715584 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.715563    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-credential-socket\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.715806 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.715568    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-data\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.717279 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.717257    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-envoy\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.717693 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.717678    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-podinfo\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.775119 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.775081    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-token\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.777584 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.777554    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-752cf\" (UniqueName: \"kubernetes.io/projected/90685ad5-9312-489e-ab58-84c12ebcc5ab-kube-api-access-752cf\") pod \"e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") " pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:27.844846 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:27.844817    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:28.282512 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:28.272794    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"]
Oct 30 19:15:28.307749 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:15:28.307716    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90685ad5_9312_489e_ab58_84c12ebcc5ab.slice/crio-07349a1975ba09a8e14f97bb95ba62415a2f982fe196ddc15a7cf9994d2260dd WatchSource:0}: Error finding container 07349a1975ba09a8e14f97bb95ba62415a2f982fe196ddc15a7cf9994d2260dd: Status 404 returned error can't find the container with id 07349a1975ba09a8e14f97bb95ba62415a2f982fe196ddc15a7cf9994d2260dd
Oct 30 19:15:28.310380 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:28.310348    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:15:28.310447 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:28.310415    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:15:28.310447 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:28.310441    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:15:28.382803 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:28.382770    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8" event={"ID":"90685ad5-9312-489e-ab58-84c12ebcc5ab","Type":"ContainerStarted","Data":"1541c711d30aa1903b3b2c837e2cf0a78a2402858490f4f01a9c25dd97ea34ec"}
Oct 30 19:15:28.382914 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:28.382840    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8" event={"ID":"90685ad5-9312-489e-ab58-84c12ebcc5ab","Type":"ContainerStarted","Data":"07349a1975ba09a8e14f97bb95ba62415a2f982fe196ddc15a7cf9994d2260dd"}
Oct 30 19:15:28.499995 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:28.499942    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8" podStartSLOduration=1.499924674 podStartE2EDuration="1.499924674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:28.497288597 +0000 UTC m=+2326.023163695" watchObservedRunningTime="2026-10-30 19:15:28.499924674 +0000 UTC m=+2326.025799766"
Oct 30 19:15:28.603924 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:28.603838    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe43caaf-eb11-4b75-a185-66ab33940e27" path="/var/lib/kubelet/pods/fe43caaf-eb11-4b75-a185-66ab33940e27/volumes"
Oct 30 19:15:28.845097 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:28.845057    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:29.851090 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:29.851059    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:30.397073 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:30.397041    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:30.398083 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:30.398059    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:15:32.551649 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:32.551588    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main" containerID="cri-o://e0b6352e47637d6a07c2d368655a1bda47525b1efb16f79732a58fc66d03fa99" gracePeriod=45
Oct 30 19:15:32.776645 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:32.774032    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"]
Oct 30 19:15:32.776645 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:32.774288    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc" podUID="ce2fa48e-5bbf-4846-8068-2c8532e3e37e" containerName="maas-api" containerID="cri-o://36af21b9b51d31b5f4064b335a6e2ccc14b33d530f54afeaceef7e4649c5df76" gracePeriod=30
Oct 30 19:15:32.922920 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:32.922876    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 19:15:33.091216 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.091187    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-kube-api-access-cw9bc\" (UniqueName: \"kubernetes.io/projected/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-kube-api-access-cw9bc\") pod \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") "
Oct 30 19:15:33.091332 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.091299    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-dshm\") pod \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") "
Oct 30 19:15:33.091383 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.091337    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-tls-certs\" (UniqueName: \"kubernetes.io/secret/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-tls-certs\") pod \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") "
Oct 30 19:15:33.091489 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.091464    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-home\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-home\") pod \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") "
Oct 30 19:15:33.091629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.091533    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-model-cache\") pod \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") "
Oct 30 19:15:33.091629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.091562    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-tmp-dir\") pod \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\" (UID: \"78d18a17-75d1-49c4-bb28-ab7e2bc5753a\") "
Oct 30 19:15:33.094386 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.094252    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-tmp-dir" pod "78d18a17-75d1-49c4-bb28-ab7e2bc5753a" (UID: "78d18a17-75d1-49c4-bb28-ab7e2bc5753a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:33.094386 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.094264    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-kube-api-access-cw9bc" pod "78d18a17-75d1-49c4-bb28-ab7e2bc5753a" (UID: "78d18a17-75d1-49c4-bb28-ab7e2bc5753a"). InnerVolumeSpecName "kube-api-access-cw9bc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:33.094554 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.094414    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-home" pod "78d18a17-75d1-49c4-bb28-ab7e2bc5753a" (UID: "78d18a17-75d1-49c4-bb28-ab7e2bc5753a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:33.094554 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.094540    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-model-cache" pod "78d18a17-75d1-49c4-bb28-ab7e2bc5753a" (UID: "78d18a17-75d1-49c4-bb28-ab7e2bc5753a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:33.095007 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.094978    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-tls-certs" pod "78d18a17-75d1-49c4-bb28-ab7e2bc5753a" (UID: "78d18a17-75d1-49c4-bb28-ab7e2bc5753a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:33.099372 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.099350    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-dshm" pod "78d18a17-75d1-49c4-bb28-ab7e2bc5753a" (UID: "78d18a17-75d1-49c4-bb28-ab7e2bc5753a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:33.108652 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.108574    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:15:33.195728 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.195690    4544 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-home\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:33.195728 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.195728    4544 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-model-cache\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:33.195979 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.195743    4544 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-tmp-dir\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:33.195979 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.195758    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cw9bc\" (UniqueName: \"kubernetes.io/projected/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-kube-api-access-cw9bc\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:33.195979 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.195774    4544 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-dshm\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:33.195979 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.195786    4544 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78d18a17-75d1-49c4-bb28-ab7e2bc5753a-tls-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:33.296483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.296449    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-openshift-service-ca\") pod \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\" (UID: \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\") "
Oct 30 19:15:33.296687 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.296642    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-kube-api-access-5ps8d\" (UniqueName: \"kubernetes.io/projected/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-kube-api-access-5ps8d\") pod \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\" (UID: \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\") "
Oct 30 19:15:33.296687 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.296686    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-maas-api-tls\") pod \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\" (UID: \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\") "
Oct 30 19:15:33.296824 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.296718    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-metrics-certs\") pod \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\" (UID: \"ce2fa48e-5bbf-4846-8068-2c8532e3e37e\") "
Oct 30 19:15:33.296883 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.296829    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-openshift-service-ca" pod "ce2fa48e-5bbf-4846-8068-2c8532e3e37e" (UID: "ce2fa48e-5bbf-4846-8068-2c8532e3e37e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:33.297200 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.297174    4544 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-openshift-service-ca\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:33.298673 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.298649    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-kube-api-access-5ps8d" pod "ce2fa48e-5bbf-4846-8068-2c8532e3e37e" (UID: "ce2fa48e-5bbf-4846-8068-2c8532e3e37e"). InnerVolumeSpecName "kube-api-access-5ps8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:33.299113 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.299092    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-maas-api-tls" pod "ce2fa48e-5bbf-4846-8068-2c8532e3e37e" (UID: "ce2fa48e-5bbf-4846-8068-2c8532e3e37e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:33.299113 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.299106    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-metrics-certs" pod "ce2fa48e-5bbf-4846-8068-2c8532e3e37e" (UID: "ce2fa48e-5bbf-4846-8068-2c8532e3e37e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:33.398414 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.398326    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ps8d\" (UniqueName: \"kubernetes.io/projected/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-kube-api-access-5ps8d\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:33.398414 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.398367    4544 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-maas-api-tls\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:33.398414 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.398385    4544 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce2fa48e-5bbf-4846-8068-2c8532e3e37e-metrics-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:33.417671 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.417599    4544 generic.go:356] "Generic (PLEG): container finished" podID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerID="e0b6352e47637d6a07c2d368655a1bda47525b1efb16f79732a58fc66d03fa99" exitCode=0
Oct 30 19:15:33.417831 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.417699    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"
Oct 30 19:15:33.417831 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.417709    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" event={"ID":"78d18a17-75d1-49c4-bb28-ab7e2bc5753a","Type":"ContainerDied","Data":"e0b6352e47637d6a07c2d368655a1bda47525b1efb16f79732a58fc66d03fa99"}
Oct 30 19:15:33.417831 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.417756    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5" event={"ID":"78d18a17-75d1-49c4-bb28-ab7e2bc5753a","Type":"ContainerDied","Data":"e0bb6cad76e4a17f67882d22ae443aaf1ab5608089af26005d4c4bec33957567"}
Oct 30 19:15:33.417831 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.417773    4544 scope.go:122] "RemoveContainer" containerID="e0b6352e47637d6a07c2d368655a1bda47525b1efb16f79732a58fc66d03fa99"
Oct 30 19:15:33.419260 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.419233    4544 generic.go:356] "Generic (PLEG): container finished" podID="ce2fa48e-5bbf-4846-8068-2c8532e3e37e" containerID="36af21b9b51d31b5f4064b335a6e2ccc14b33d530f54afeaceef7e4649c5df76" exitCode=0
Oct 30 19:15:33.419375 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.419278    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc" event={"ID":"ce2fa48e-5bbf-4846-8068-2c8532e3e37e","Type":"ContainerDied","Data":"36af21b9b51d31b5f4064b335a6e2ccc14b33d530f54afeaceef7e4649c5df76"}
Oct 30 19:15:33.419375 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.419306    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"
Oct 30 19:15:33.419375 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.419365    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc" event={"ID":"ce2fa48e-5bbf-4846-8068-2c8532e3e37e","Type":"ContainerDied","Data":"f2bfada5ad20d3cd9456532e4f074a132ea49bac5f7e56cfe59494e275b2188f"}
Oct 30 19:15:33.429716 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.429693    4544 scope.go:122] "RemoveContainer" containerID="e0b6352e47637d6a07c2d368655a1bda47525b1efb16f79732a58fc66d03fa99"
Oct 30 19:15:33.430081 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:15:33.429962    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0b6352e47637d6a07c2d368655a1bda47525b1efb16f79732a58fc66d03fa99\": container with ID starting with e0b6352e47637d6a07c2d368655a1bda47525b1efb16f79732a58fc66d03fa99 not found: ID does not exist" containerID="e0b6352e47637d6a07c2d368655a1bda47525b1efb16f79732a58fc66d03fa99"
Oct 30 19:15:33.430081 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.429986    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0b6352e47637d6a07c2d368655a1bda47525b1efb16f79732a58fc66d03fa99"} err="failed to get container status \"e0b6352e47637d6a07c2d368655a1bda47525b1efb16f79732a58fc66d03fa99\": rpc error: code = NotFound desc = could not find container \"e0b6352e47637d6a07c2d368655a1bda47525b1efb16f79732a58fc66d03fa99\": container with ID starting with e0b6352e47637d6a07c2d368655a1bda47525b1efb16f79732a58fc66d03fa99 not found: ID does not exist"
Oct 30 19:15:33.430081 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.430011    4544 scope.go:122] "RemoveContainer" containerID="36af21b9b51d31b5f4064b335a6e2ccc14b33d530f54afeaceef7e4649c5df76"
Oct 30 19:15:33.439074 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.439049    4544 scope.go:122] "RemoveContainer" containerID="36af21b9b51d31b5f4064b335a6e2ccc14b33d530f54afeaceef7e4649c5df76"
Oct 30 19:15:33.439341 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:15:33.439320    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36af21b9b51d31b5f4064b335a6e2ccc14b33d530f54afeaceef7e4649c5df76\": container with ID starting with 36af21b9b51d31b5f4064b335a6e2ccc14b33d530f54afeaceef7e4649c5df76 not found: ID does not exist" containerID="36af21b9b51d31b5f4064b335a6e2ccc14b33d530f54afeaceef7e4649c5df76"
Oct 30 19:15:33.439421 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.439347    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36af21b9b51d31b5f4064b335a6e2ccc14b33d530f54afeaceef7e4649c5df76"} err="failed to get container status \"36af21b9b51d31b5f4064b335a6e2ccc14b33d530f54afeaceef7e4649c5df76\": rpc error: code = NotFound desc = could not find container \"36af21b9b51d31b5f4064b335a6e2ccc14b33d530f54afeaceef7e4649c5df76\": container with ID starting with 36af21b9b51d31b5f4064b335a6e2ccc14b33d530f54afeaceef7e4649c5df76 not found: ID does not exist"
Oct 30 19:15:33.594777 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.591544    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"]
Oct 30 19:15:33.594777 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.591616    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-5f80ac/premium-w1-5f80ac-kserve-58776b59c6-m57l5"]
Oct 30 19:15:33.696216 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.696083    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"]
Oct 30 19:15:33.702989 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:33.702959    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2ef49a-7df8bb687c-qpwjc"]
Oct 30 19:15:34.604116 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:34.604079    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" path="/var/lib/kubelet/pods/78d18a17-75d1-49c4-bb28-ab7e2bc5753a/volumes"
Oct 30 19:15:34.604724 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:34.604705    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce2fa48e-5bbf-4846-8068-2c8532e3e37e" path="/var/lib/kubelet/pods/ce2fa48e-5bbf-4846-8068-2c8532e3e37e/volumes"
Oct 30 19:15:35.532019 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:15:35.531985    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:15:39.345073 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.345037    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt"]
Oct 30 19:15:39.350952 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.345787    4544 cpu_manager.go:414] "containerMap: removing container" podUID="ce2fa48e-5bbf-4846-8068-2c8532e3e37e" containerName="maas-api"
Oct 30 19:15:39.350952 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.345803    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce2fa48e-5bbf-4846-8068-2c8532e3e37e" containerName="maas-api"
Oct 30 19:15:39.350952 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.345844    4544 cpu_manager.go:414] "containerMap: removing container" podUID="8403daa2-a502-4d25-aea1-0558f21d6e7b" containerName="cleanup"
Oct 30 19:15:39.350952 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.345851    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8403daa2-a502-4d25-aea1-0558f21d6e7b" containerName="cleanup"
Oct 30 19:15:39.350952 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.345865    4544 cpu_manager.go:414] "containerMap: removing container" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main"
Oct 30 19:15:39.350952 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.345872    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main"
Oct 30 19:15:39.350952 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.345884    4544 cpu_manager.go:414] "containerMap: removing container" podUID="6652a8f9-4c8c-43f2-be7f-9f8025690485" containerName="cleanup"
Oct 30 19:15:39.350952 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.345891    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6652a8f9-4c8c-43f2-be7f-9f8025690485" containerName="cleanup"
Oct 30 19:15:39.350952 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.346008    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3c4f1eb-a61c-4184-8ae6-afc8a54e024e" containerName="istio-proxy"
Oct 30 19:15:39.350952 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.346023    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="78d18a17-75d1-49c4-bb28-ab7e2bc5753a" containerName="main"
Oct 30 19:15:39.350952 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.346034    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="6652a8f9-4c8c-43f2-be7f-9f8025690485" containerName="cleanup"
Oct 30 19:15:39.350952 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.346048    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce2fa48e-5bbf-4846-8068-2c8532e3e37e" containerName="maas-api"
Oct 30 19:15:39.351427 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.351410    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt"
Oct 30 19:15:39.395630 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.395572    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aed6caf3-296f-46b3-af0d-fd8712434652-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt\" (UID: \"aed6caf3-296f-46b3-af0d-fd8712434652\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt"
Oct 30 19:15:39.395931 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.395906    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsxdp\" (UniqueName: \"kubernetes.io/projected/aed6caf3-296f-46b3-af0d-fd8712434652-kube-api-access-hsxdp\") pod \"payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt\" (UID: \"aed6caf3-296f-46b3-af0d-fd8712434652\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt"
Oct 30 19:15:39.411054 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.411016    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt"]
Oct 30 19:15:39.468407 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.454935    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-29de7b\""
Oct 30 19:15:39.470748 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.470707    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-29de7b-dockercfg-9qbnw\""
Oct 30 19:15:39.497306 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.497275    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aed6caf3-296f-46b3-af0d-fd8712434652-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt\" (UID: \"aed6caf3-296f-46b3-af0d-fd8712434652\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt"
Oct 30 19:15:39.497495 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.497316    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsxdp\" (UniqueName: \"kubernetes.io/projected/aed6caf3-296f-46b3-af0d-fd8712434652-kube-api-access-hsxdp\") pod \"payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt\" (UID: \"aed6caf3-296f-46b3-af0d-fd8712434652\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt"
Oct 30 19:15:39.497989 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.497964    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aed6caf3-296f-46b3-af0d-fd8712434652-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt\" (UID: \"aed6caf3-296f-46b3-af0d-fd8712434652\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt"
Oct 30 19:15:39.505993 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.505955    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls"]
Oct 30 19:15:39.512957 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.512936    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls"
Oct 30 19:15:39.551203 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.551179    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsxdp\" (UniqueName: \"kubernetes.io/projected/aed6caf3-296f-46b3-af0d-fd8712434652-kube-api-access-hsxdp\") pod \"payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt\" (UID: \"aed6caf3-296f-46b3-af0d-fd8712434652\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt"
Oct 30 19:15:39.564362 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.564335    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls"]
Oct 30 19:15:39.598196 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.598111    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a90f74a2-102e-4495-acc5-069e44aa693d-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls\" (UID: \"a90f74a2-102e-4495-acc5-069e44aa693d\") " pod="openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls"
Oct 30 19:15:39.598343 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.598285    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrhxv\" (UniqueName: \"kubernetes.io/projected/a90f74a2-102e-4495-acc5-069e44aa693d-kube-api-access-mrhxv\") pod \"payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls\" (UID: \"a90f74a2-102e-4495-acc5-069e44aa693d\") " pod="openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls"
Oct 30 19:15:39.670426 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.670401    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt"
Oct 30 19:15:39.699679 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.699648    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrhxv\" (UniqueName: \"kubernetes.io/projected/a90f74a2-102e-4495-acc5-069e44aa693d-kube-api-access-mrhxv\") pod \"payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls\" (UID: \"a90f74a2-102e-4495-acc5-069e44aa693d\") " pod="openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls"
Oct 30 19:15:39.699804 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.699755    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a90f74a2-102e-4495-acc5-069e44aa693d-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls\" (UID: \"a90f74a2-102e-4495-acc5-069e44aa693d\") " pod="openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls"
Oct 30 19:15:39.700443 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.700423    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a90f74a2-102e-4495-acc5-069e44aa693d-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls\" (UID: \"a90f74a2-102e-4495-acc5-069e44aa693d\") " pod="openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls"
Oct 30 19:15:39.759175 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.759144    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrhxv\" (UniqueName: \"kubernetes.io/projected/a90f74a2-102e-4495-acc5-069e44aa693d-kube-api-access-mrhxv\") pod \"payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls\" (UID: \"a90f74a2-102e-4495-acc5-069e44aa693d\") " pod="openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls"
Oct 30 19:15:39.827324 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:39.827286    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls"
Oct 30 19:15:40.042247 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:40.042218    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt"]
Oct 30 19:15:40.044179 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:15:40.044151    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaed6caf3_296f_46b3_af0d_fd8712434652.slice/crio-19f573785003f2a5c2a5a5593084914b12206801897b80665e2309cf0cf8d554 WatchSource:0}: Error finding container 19f573785003f2a5c2a5a5593084914b12206801897b80665e2309cf0cf8d554: Status 404 returned error can't find the container with id 19f573785003f2a5c2a5a5593084914b12206801897b80665e2309cf0cf8d554
Oct 30 19:15:40.213124 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:40.213099    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls"]
Oct 30 19:15:40.215514 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:15:40.215486    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda90f74a2_102e_4495_acc5_069e44aa693d.slice/crio-7a81c105f12af329c5c55a52d2f041ab91ebb61f0392bb04e85bec039bb99343 WatchSource:0}: Error finding container 7a81c105f12af329c5c55a52d2f041ab91ebb61f0392bb04e85bec039bb99343: Status 404 returned error can't find the container with id 7a81c105f12af329c5c55a52d2f041ab91ebb61f0392bb04e85bec039bb99343
Oct 30 19:15:40.475178 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:40.475075    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt" event={"ID":"aed6caf3-296f-46b3-af0d-fd8712434652","Type":"ContainerStarted","Data":"4fe38872d0cad6c92faef4baa00cc91b8348c5a1e53c84c9c5ebb4eb4a0b9e60"}
Oct 30 19:15:40.475178 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:40.475137    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt" event={"ID":"aed6caf3-296f-46b3-af0d-fd8712434652","Type":"ContainerStarted","Data":"19f573785003f2a5c2a5a5593084914b12206801897b80665e2309cf0cf8d554"}
Oct 30 19:15:40.475178 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:40.475161    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt"
Oct 30 19:15:40.476857 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:40.476826    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls" event={"ID":"a90f74a2-102e-4495-acc5-069e44aa693d","Type":"ContainerStarted","Data":"2b8e1be8fb00586b851e1132ef9a1e6b156eb9448a5fbbb5e7e9a7645c519b88"}
Oct 30 19:15:40.476974 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:40.476902    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls"
Oct 30 19:15:40.476974 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:40.476927    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls" event={"ID":"a90f74a2-102e-4495-acc5-069e44aa693d","Type":"ContainerStarted","Data":"7a81c105f12af329c5c55a52d2f041ab91ebb61f0392bb04e85bec039bb99343"}
Oct 30 19:15:40.586148 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:40.586089    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt" podStartSLOduration=1.586071845 podStartE2EDuration="1.586071845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:40.581960263 +0000 UTC m=+2338.107835364" watchObservedRunningTime="2026-10-30 19:15:40.586071845 +0000 UTC m=+2338.111946942"
Oct 30 19:15:40.690790 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:40.690740    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls" podStartSLOduration=1.690726493 podStartE2EDuration="1.690726493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:40.68773868 +0000 UTC m=+2338.213613776" watchObservedRunningTime="2026-10-30 19:15:40.690726493 +0000 UTC m=+2338.216601588"
Oct 30 19:15:43.543689 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:43.543646    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"]
Oct 30 19:15:43.544066 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:43.543862    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb" podUID="729b4f97-7d22-49ae-ac1b-4528af3b5242" containerName="istio-proxy" containerID="cri-o://df2df1729f7deaa0eea3675f7c547382ac574c5107500bb4ea9c12b0f2003130" gracePeriod=30
Oct 30 19:15:43.705014 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:43.704896    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"]
Oct 30 19:15:48.800096 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.800063    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:15:48.930413 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.930334    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/729b4f97-7d22-49ae-ac1b-4528af3b5242-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/729b4f97-7d22-49ae-ac1b-4528af3b5242-istiod-ca-cert\") pod \"729b4f97-7d22-49ae-ac1b-4528af3b5242\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") "
Oct 30 19:15:48.930413 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.930390    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-workload-certs\") pod \"729b4f97-7d22-49ae-ac1b-4528af3b5242\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") "
Oct 30 19:15:48.930413 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.930412    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-token\" (UniqueName: \"kubernetes.io/projected/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-token\") pod \"729b4f97-7d22-49ae-ac1b-4528af3b5242\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") "
Oct 30 19:15:48.930742 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.930485    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-credential-socket\") pod \"729b4f97-7d22-49ae-ac1b-4528af3b5242\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") "
Oct 30 19:15:48.930742 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.930508    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-envoy\") pod \"729b4f97-7d22-49ae-ac1b-4528af3b5242\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") "
Oct 30 19:15:48.930742 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.930525    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-podinfo\") pod \"729b4f97-7d22-49ae-ac1b-4528af3b5242\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") "
Oct 30 19:15:48.930742 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.930552    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-data\") pod \"729b4f97-7d22-49ae-ac1b-4528af3b5242\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") "
Oct 30 19:15:48.930742 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.930580    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/729b4f97-7d22-49ae-ac1b-4528af3b5242-kube-api-access-p7xxp\" (UniqueName: \"kubernetes.io/projected/729b4f97-7d22-49ae-ac1b-4528af3b5242-kube-api-access-p7xxp\") pod \"729b4f97-7d22-49ae-ac1b-4528af3b5242\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") "
Oct 30 19:15:48.930742 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.930693    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-workload-socket\") pod \"729b4f97-7d22-49ae-ac1b-4528af3b5242\" (UID: \"729b4f97-7d22-49ae-ac1b-4528af3b5242\") "
Oct 30 19:15:48.930742 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.930726    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-workload-certs" pod "729b4f97-7d22-49ae-ac1b-4528af3b5242" (UID: "729b4f97-7d22-49ae-ac1b-4528af3b5242"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:48.931109 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.930784    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/729b4f97-7d22-49ae-ac1b-4528af3b5242-istiod-ca-cert" pod "729b4f97-7d22-49ae-ac1b-4528af3b5242" (UID: "729b4f97-7d22-49ae-ac1b-4528af3b5242"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:48.931109 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.930940    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-data" pod "729b4f97-7d22-49ae-ac1b-4528af3b5242" (UID: "729b4f97-7d22-49ae-ac1b-4528af3b5242"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:48.931109 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.931085    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-credential-socket" pod "729b4f97-7d22-49ae-ac1b-4528af3b5242" (UID: "729b4f97-7d22-49ae-ac1b-4528af3b5242"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:48.931250 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.931118    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-workload-socket" pod "729b4f97-7d22-49ae-ac1b-4528af3b5242" (UID: "729b4f97-7d22-49ae-ac1b-4528af3b5242"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:48.931250 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.931228    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:48.931341 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.931248    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:48.931341 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.931263    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:48.931341 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.931277    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/729b4f97-7d22-49ae-ac1b-4528af3b5242-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:48.931341 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.931292    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:48.932900 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.932879    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-token" pod "729b4f97-7d22-49ae-ac1b-4528af3b5242" (UID: "729b4f97-7d22-49ae-ac1b-4528af3b5242"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:48.932977 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.932898    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/729b4f97-7d22-49ae-ac1b-4528af3b5242-kube-api-access-p7xxp" pod "729b4f97-7d22-49ae-ac1b-4528af3b5242" (UID: "729b4f97-7d22-49ae-ac1b-4528af3b5242"). InnerVolumeSpecName "kube-api-access-p7xxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:48.933103 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.933088    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-envoy" pod "729b4f97-7d22-49ae-ac1b-4528af3b5242" (UID: "729b4f97-7d22-49ae-ac1b-4528af3b5242"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:48.933178 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:48.933157    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-podinfo" pod "729b4f97-7d22-49ae-ac1b-4528af3b5242" (UID: "729b4f97-7d22-49ae-ac1b-4528af3b5242"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:15:49.032188 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:49.032159    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:49.032188 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:49.032186    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:49.032188 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:49.032198    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/729b4f97-7d22-49ae-ac1b-4528af3b5242-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:49.032399 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:49.032207    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7xxp\" (UniqueName: \"kubernetes.io/projected/729b4f97-7d22-49ae-ac1b-4528af3b5242-kube-api-access-p7xxp\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:49.541116 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:49.541079    4544 generic.go:356] "Generic (PLEG): container finished" podID="729b4f97-7d22-49ae-ac1b-4528af3b5242" containerID="df2df1729f7deaa0eea3675f7c547382ac574c5107500bb4ea9c12b0f2003130" exitCode=0
Oct 30 19:15:49.541290 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:49.541124    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb" event={"ID":"729b4f97-7d22-49ae-ac1b-4528af3b5242","Type":"ContainerDied","Data":"df2df1729f7deaa0eea3675f7c547382ac574c5107500bb4ea9c12b0f2003130"}
Oct 30 19:15:49.541290 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:49.541134    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"
Oct 30 19:15:49.541290 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:49.541177    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb" event={"ID":"729b4f97-7d22-49ae-ac1b-4528af3b5242","Type":"ContainerDied","Data":"3a1c72389702dd4813029c8712460795bd9bc9d819c7be15da8e474f86aca35b"}
Oct 30 19:15:49.541290 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:49.541205    4544 scope.go:122] "RemoveContainer" containerID="df2df1729f7deaa0eea3675f7c547382ac574c5107500bb4ea9c12b0f2003130"
Oct 30 19:15:49.551088 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:49.551071    4544 scope.go:122] "RemoveContainer" containerID="571a3de3132365c3dc0b58d04e53ce7ad88e4c525ed948d1fabac6985a99a850"
Oct 30 19:15:49.559204 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:49.559181    4544 scope.go:122] "RemoveContainer" containerID="df2df1729f7deaa0eea3675f7c547382ac574c5107500bb4ea9c12b0f2003130"
Oct 30 19:15:49.559487 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:15:49.559465    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df2df1729f7deaa0eea3675f7c547382ac574c5107500bb4ea9c12b0f2003130\": container with ID starting with df2df1729f7deaa0eea3675f7c547382ac574c5107500bb4ea9c12b0f2003130 not found: ID does not exist" containerID="df2df1729f7deaa0eea3675f7c547382ac574c5107500bb4ea9c12b0f2003130"
Oct 30 19:15:49.559546 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:49.559499    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df2df1729f7deaa0eea3675f7c547382ac574c5107500bb4ea9c12b0f2003130"} err="failed to get container status \"df2df1729f7deaa0eea3675f7c547382ac574c5107500bb4ea9c12b0f2003130\": rpc error: code = NotFound desc = could not find container \"df2df1729f7deaa0eea3675f7c547382ac574c5107500bb4ea9c12b0f2003130\": container with ID starting with df2df1729f7deaa0eea3675f7c547382ac574c5107500bb4ea9c12b0f2003130 not found: ID does not exist"
Oct 30 19:15:49.559546 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:49.559529    4544 scope.go:122] "RemoveContainer" containerID="571a3de3132365c3dc0b58d04e53ce7ad88e4c525ed948d1fabac6985a99a850"
Oct 30 19:15:49.559824 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:15:49.559801    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"571a3de3132365c3dc0b58d04e53ce7ad88e4c525ed948d1fabac6985a99a850\": container with ID starting with 571a3de3132365c3dc0b58d04e53ce7ad88e4c525ed948d1fabac6985a99a850 not found: ID does not exist" containerID="571a3de3132365c3dc0b58d04e53ce7ad88e4c525ed948d1fabac6985a99a850"
Oct 30 19:15:49.559893 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:49.559830    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"571a3de3132365c3dc0b58d04e53ce7ad88e4c525ed948d1fabac6985a99a850"} err="failed to get container status \"571a3de3132365c3dc0b58d04e53ce7ad88e4c525ed948d1fabac6985a99a850\": rpc error: code = NotFound desc = could not find container \"571a3de3132365c3dc0b58d04e53ce7ad88e4c525ed948d1fabac6985a99a850\": container with ID starting with 571a3de3132365c3dc0b58d04e53ce7ad88e4c525ed948d1fabac6985a99a850 not found: ID does not exist"
Oct 30 19:15:49.698934 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:49.698892    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"]
Oct 30 19:15:49.745297 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:49.745268    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-2ef49a-openshift-default-67f94c749f-6j8jb"]
Oct 30 19:15:50.602130 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:50.602098    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="729b4f97-7d22-49ae-ac1b-4528af3b5242" path="/var/lib/kubelet/pods/729b4f97-7d22-49ae-ac1b-4528af3b5242/volumes"
Oct 30 19:15:51.485779 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:51.485751    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls"
Oct 30 19:15:51.485985 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:51.485970    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt"
Oct 30 19:15:58.731492 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:58.731425    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main" containerID="cri-o://12fd8132f4acf9a43d7ca404706a45331655e119bfa08d85dd42aa68e139e905" gracePeriod=45
Oct 30 19:15:58.994867 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:58.994808    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 19:15:59.044359 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.044327    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-tmp-dir\") pod \"f71923a7-7e8a-48e0-adcd-a061c07eb212\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") "
Oct 30 19:15:59.044505 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.044365    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-dshm\") pod \"f71923a7-7e8a-48e0-adcd-a061c07eb212\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") "
Oct 30 19:15:59.044505 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.044421    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f71923a7-7e8a-48e0-adcd-a061c07eb212-kube-api-access-sv8pt\" (UniqueName: \"kubernetes.io/projected/f71923a7-7e8a-48e0-adcd-a061c07eb212-kube-api-access-sv8pt\") pod \"f71923a7-7e8a-48e0-adcd-a061c07eb212\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") "
Oct 30 19:15:59.044505 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.044453    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-model-cache\") pod \"f71923a7-7e8a-48e0-adcd-a061c07eb212\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") "
Oct 30 19:15:59.044730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.044527    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f71923a7-7e8a-48e0-adcd-a061c07eb212-tls-certs\" (UniqueName: \"kubernetes.io/secret/f71923a7-7e8a-48e0-adcd-a061c07eb212-tls-certs\") pod \"f71923a7-7e8a-48e0-adcd-a061c07eb212\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") "
Oct 30 19:15:59.044730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.044598    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-tmp-dir" pod "f71923a7-7e8a-48e0-adcd-a061c07eb212" (UID: "f71923a7-7e8a-48e0-adcd-a061c07eb212"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:59.044730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.044711    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-home\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-home\") pod \"f71923a7-7e8a-48e0-adcd-a061c07eb212\" (UID: \"f71923a7-7e8a-48e0-adcd-a061c07eb212\") "
Oct 30 19:15:59.044885 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.044859    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-model-cache" pod "f71923a7-7e8a-48e0-adcd-a061c07eb212" (UID: "f71923a7-7e8a-48e0-adcd-a061c07eb212"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:59.045085 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.045065    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-home" pod "f71923a7-7e8a-48e0-adcd-a061c07eb212" (UID: "f71923a7-7e8a-48e0-adcd-a061c07eb212"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:59.045188 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.045174    4544 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-home\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:59.045245 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.045192    4544 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-tmp-dir\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:59.045245 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.045202    4544 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-model-cache\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:59.046790 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.046767    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-dshm" pod "f71923a7-7e8a-48e0-adcd-a061c07eb212" (UID: "f71923a7-7e8a-48e0-adcd-a061c07eb212"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:59.046903 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.046824    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f71923a7-7e8a-48e0-adcd-a061c07eb212-tls-certs" pod "f71923a7-7e8a-48e0-adcd-a061c07eb212" (UID: "f71923a7-7e8a-48e0-adcd-a061c07eb212"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:59.046966 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.046946    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f71923a7-7e8a-48e0-adcd-a061c07eb212-kube-api-access-sv8pt" pod "f71923a7-7e8a-48e0-adcd-a061c07eb212" (UID: "f71923a7-7e8a-48e0-adcd-a061c07eb212"). InnerVolumeSpecName "kube-api-access-sv8pt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:59.146404 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.146368    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sv8pt\" (UniqueName: \"kubernetes.io/projected/f71923a7-7e8a-48e0-adcd-a061c07eb212-kube-api-access-sv8pt\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:59.146404 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.146400    4544 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f71923a7-7e8a-48e0-adcd-a061c07eb212-tls-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:59.146404 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.146410    4544 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f71923a7-7e8a-48e0-adcd-a061c07eb212-dshm\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:59.601162 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.601127    4544 generic.go:356] "Generic (PLEG): container finished" podID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerID="12fd8132f4acf9a43d7ca404706a45331655e119bfa08d85dd42aa68e139e905" exitCode=0
Oct 30 19:15:59.601377 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.601170    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"
Oct 30 19:15:59.601377 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.601219    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" event={"ID":"f71923a7-7e8a-48e0-adcd-a061c07eb212","Type":"ContainerDied","Data":"12fd8132f4acf9a43d7ca404706a45331655e119bfa08d85dd42aa68e139e905"}
Oct 30 19:15:59.601377 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.601265    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482" event={"ID":"f71923a7-7e8a-48e0-adcd-a061c07eb212","Type":"ContainerDied","Data":"b2058c2e748453c5d7f932c924363b3a9ddafc59e55e4c70c6ef332707807ef7"}
Oct 30 19:15:59.601377 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.601287    4544 scope.go:122] "RemoveContainer" containerID="12fd8132f4acf9a43d7ca404706a45331655e119bfa08d85dd42aa68e139e905"
Oct 30 19:15:59.611037 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.611019    4544 scope.go:122] "RemoveContainer" containerID="12fd8132f4acf9a43d7ca404706a45331655e119bfa08d85dd42aa68e139e905"
Oct 30 19:15:59.611306 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:15:59.611284    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12fd8132f4acf9a43d7ca404706a45331655e119bfa08d85dd42aa68e139e905\": container with ID starting with 12fd8132f4acf9a43d7ca404706a45331655e119bfa08d85dd42aa68e139e905 not found: ID does not exist" containerID="12fd8132f4acf9a43d7ca404706a45331655e119bfa08d85dd42aa68e139e905"
Oct 30 19:15:59.611364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.611314    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12fd8132f4acf9a43d7ca404706a45331655e119bfa08d85dd42aa68e139e905"} err="failed to get container status \"12fd8132f4acf9a43d7ca404706a45331655e119bfa08d85dd42aa68e139e905\": rpc error: code = NotFound desc = could not find container \"12fd8132f4acf9a43d7ca404706a45331655e119bfa08d85dd42aa68e139e905\": container with ID starting with 12fd8132f4acf9a43d7ca404706a45331655e119bfa08d85dd42aa68e139e905 not found: ID does not exist"
Oct 30 19:15:59.759893 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.759859    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"]
Oct 30 19:15:59.806216 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:15:59.806184    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-a1617e/simulator-w3-a1617e-kserve-8576757679-9k482"]
Oct 30 19:16:00.602639 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:00.602588    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" path="/var/lib/kubelet/pods/f71923a7-7e8a-48e0-adcd-a061c07eb212/volumes"
Oct 30 19:16:05.531183 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:16:05.531146    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:16:11.389716 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.389672    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"]
Oct 30 19:16:11.390280 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.389923    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l" podUID="15b9134c-0ca4-4285-a041-badc8cddc776" containerName="maas-api" containerID="cri-o://2a3d6f377ab168f1bfddad06f2abaab8934c523d1cf1de55a23218b22b3860f1" gracePeriod=30
Oct 30 19:16:11.667203 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.667175    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 19:16:11.671573 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.671541    4544 generic.go:356] "Generic (PLEG): container finished" podID="15b9134c-0ca4-4285-a041-badc8cddc776" containerID="2a3d6f377ab168f1bfddad06f2abaab8934c523d1cf1de55a23218b22b3860f1" exitCode=0
Oct 30 19:16:11.671759 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.671577    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"
Oct 30 19:16:11.671759 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.671650    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l" event={"ID":"15b9134c-0ca4-4285-a041-badc8cddc776","Type":"ContainerDied","Data":"2a3d6f377ab168f1bfddad06f2abaab8934c523d1cf1de55a23218b22b3860f1"}
Oct 30 19:16:11.671759 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.671699    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l" event={"ID":"15b9134c-0ca4-4285-a041-badc8cddc776","Type":"ContainerDied","Data":"2f5b063cdbc0d9c2cbc55d4bf6ea0487c50ce6055e9e180cc0b0b52d00481487"}
Oct 30 19:16:11.671759 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.671728    4544 scope.go:122] "RemoveContainer" containerID="2a3d6f377ab168f1bfddad06f2abaab8934c523d1cf1de55a23218b22b3860f1"
Oct 30 19:16:11.680499 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.680479    4544 scope.go:122] "RemoveContainer" containerID="2a3d6f377ab168f1bfddad06f2abaab8934c523d1cf1de55a23218b22b3860f1"
Oct 30 19:16:11.680808 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:16:11.680784    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a3d6f377ab168f1bfddad06f2abaab8934c523d1cf1de55a23218b22b3860f1\": container with ID starting with 2a3d6f377ab168f1bfddad06f2abaab8934c523d1cf1de55a23218b22b3860f1 not found: ID does not exist" containerID="2a3d6f377ab168f1bfddad06f2abaab8934c523d1cf1de55a23218b22b3860f1"
Oct 30 19:16:11.680894 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.680821    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a3d6f377ab168f1bfddad06f2abaab8934c523d1cf1de55a23218b22b3860f1"} err="failed to get container status \"2a3d6f377ab168f1bfddad06f2abaab8934c523d1cf1de55a23218b22b3860f1\": rpc error: code = NotFound desc = could not find container \"2a3d6f377ab168f1bfddad06f2abaab8934c523d1cf1de55a23218b22b3860f1\": container with ID starting with 2a3d6f377ab168f1bfddad06f2abaab8934c523d1cf1de55a23218b22b3860f1 not found: ID does not exist"
Oct 30 19:16:11.798411 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.798350    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/15b9134c-0ca4-4285-a041-badc8cddc776-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15b9134c-0ca4-4285-a041-badc8cddc776-maas-api-tls\") pod \"15b9134c-0ca4-4285-a041-badc8cddc776\" (UID: \"15b9134c-0ca4-4285-a041-badc8cddc776\") "
Oct 30 19:16:11.801629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.798864    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15b9134c-0ca4-4285-a041-badc8cddc776-kube-api-access-5hxrl\" (UniqueName: \"kubernetes.io/projected/15b9134c-0ca4-4285-a041-badc8cddc776-kube-api-access-5hxrl\") pod \"15b9134c-0ca4-4285-a041-badc8cddc776\" (UID: \"15b9134c-0ca4-4285-a041-badc8cddc776\") "
Oct 30 19:16:11.801629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.798950    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/15b9134c-0ca4-4285-a041-badc8cddc776-metrics-certs\" (UniqueName: \"kubernetes.io/secret/15b9134c-0ca4-4285-a041-badc8cddc776-metrics-certs\") pod \"15b9134c-0ca4-4285-a041-badc8cddc776\" (UID: \"15b9134c-0ca4-4285-a041-badc8cddc776\") "
Oct 30 19:16:11.801629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.799076    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/15b9134c-0ca4-4285-a041-badc8cddc776-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15b9134c-0ca4-4285-a041-badc8cddc776-openshift-service-ca\") pod \"15b9134c-0ca4-4285-a041-badc8cddc776\" (UID: \"15b9134c-0ca4-4285-a041-badc8cddc776\") "
Oct 30 19:16:11.801629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.800049    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15b9134c-0ca4-4285-a041-badc8cddc776-openshift-service-ca" pod "15b9134c-0ca4-4285-a041-badc8cddc776" (UID: "15b9134c-0ca4-4285-a041-badc8cddc776"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:11.805069 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.805038    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15b9134c-0ca4-4285-a041-badc8cddc776-metrics-certs" pod "15b9134c-0ca4-4285-a041-badc8cddc776" (UID: "15b9134c-0ca4-4285-a041-badc8cddc776"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:11.805724 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.805694    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15b9134c-0ca4-4285-a041-badc8cddc776-kube-api-access-5hxrl" pod "15b9134c-0ca4-4285-a041-badc8cddc776" (UID: "15b9134c-0ca4-4285-a041-badc8cddc776"). InnerVolumeSpecName "kube-api-access-5hxrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:11.813070 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.813037    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15b9134c-0ca4-4285-a041-badc8cddc776-maas-api-tls" pod "15b9134c-0ca4-4285-a041-badc8cddc776" (UID: "15b9134c-0ca4-4285-a041-badc8cddc776"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:11.901256 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.901180    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hxrl\" (UniqueName: \"kubernetes.io/projected/15b9134c-0ca4-4285-a041-badc8cddc776-kube-api-access-5hxrl\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:11.901256 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.901211    4544 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15b9134c-0ca4-4285-a041-badc8cddc776-metrics-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:11.901256 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.901221    4544 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15b9134c-0ca4-4285-a041-badc8cddc776-openshift-service-ca\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:11.901256 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:11.901229    4544 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15b9134c-0ca4-4285-a041-badc8cddc776-maas-api-tls\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:12.098706 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:12.098674    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"]
Oct 30 19:16:12.127388 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:12.127359    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-649d16-7b8c49469d-2zv8l"]
Oct 30 19:16:12.604284 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:12.604248    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="15b9134c-0ca4-4285-a041-badc8cddc776" path="/var/lib/kubelet/pods/15b9134c-0ca4-4285-a041-badc8cddc776/volumes"
Oct 30 19:16:22.650640 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:22.650568    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"]
Oct 30 19:16:22.651116 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:22.650816    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg" podUID="34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" containerName="istio-proxy" containerID="cri-o://9bd2c6d9961d184842235dfcc10cea15730d9af3adb6ebfcb330e0ceafcba44f" gracePeriod=30
Oct 30 19:16:24.377669 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.377623    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"]
Oct 30 19:16:24.378061 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.377842    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk" podUID="fbd9d51e-133f-4b3a-b592-38ca4b2b5d06" containerName="maas-api" containerID="cri-o://b7a60343f435000d1f0f372121979f65523d9daab7970d7ae461801bddda83d8" gracePeriod=30
Oct 30 19:16:24.651849 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.651821    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 19:16:24.673089 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.673058    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-kube-api-access-465p2\" (UniqueName: \"kubernetes.io/projected/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-kube-api-access-465p2\") pod \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\" (UID: \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\") "
Oct 30 19:16:24.673233 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.673206    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-openshift-service-ca\") pod \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\" (UID: \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\") "
Oct 30 19:16:24.673321 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.673251    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-metrics-certs\") pod \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\" (UID: \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\") "
Oct 30 19:16:24.673321 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.673278    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-maas-api-tls\") pod \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\" (UID: \"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06\") "
Oct 30 19:16:24.673632 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.673593    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-openshift-service-ca" pod "fbd9d51e-133f-4b3a-b592-38ca4b2b5d06" (UID: "fbd9d51e-133f-4b3a-b592-38ca4b2b5d06"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:24.673761 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.673745    4544 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-openshift-service-ca\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:24.675352 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.675324    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-metrics-certs" pod "fbd9d51e-133f-4b3a-b592-38ca4b2b5d06" (UID: "fbd9d51e-133f-4b3a-b592-38ca4b2b5d06"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:24.675461 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.675345    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-maas-api-tls" pod "fbd9d51e-133f-4b3a-b592-38ca4b2b5d06" (UID: "fbd9d51e-133f-4b3a-b592-38ca4b2b5d06"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:24.675461 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.675410    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-kube-api-access-465p2" pod "fbd9d51e-133f-4b3a-b592-38ca4b2b5d06" (UID: "fbd9d51e-133f-4b3a-b592-38ca4b2b5d06"). InnerVolumeSpecName "kube-api-access-465p2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:24.788647 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.776864    4544 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-metrics-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:24.788647 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.787830    4544 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-maas-api-tls\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:24.788647 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.787488    4544 generic.go:356] "Generic (PLEG): container finished" podID="fbd9d51e-133f-4b3a-b592-38ca4b2b5d06" containerID="b7a60343f435000d1f0f372121979f65523d9daab7970d7ae461801bddda83d8" exitCode=0
Oct 30 19:16:24.788647 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.787648    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"
Oct 30 19:16:24.790540 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.787534    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk" event={"ID":"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06","Type":"ContainerDied","Data":"b7a60343f435000d1f0f372121979f65523d9daab7970d7ae461801bddda83d8"}
Oct 30 19:16:24.792841 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.792804    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk" event={"ID":"fbd9d51e-133f-4b3a-b592-38ca4b2b5d06","Type":"ContainerDied","Data":"ffe054aca6a4cfb8b577a8e2a2e315b9a8ba23a9956aa9c79a5b765f705ca22f"}
Oct 30 19:16:24.792979 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.792860    4544 scope.go:122] "RemoveContainer" containerID="b7a60343f435000d1f0f372121979f65523d9daab7970d7ae461801bddda83d8"
Oct 30 19:16:24.793045 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.792997    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-465p2\" (UniqueName: \"kubernetes.io/projected/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06-kube-api-access-465p2\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:24.806056 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.806034    4544 scope.go:122] "RemoveContainer" containerID="b7a60343f435000d1f0f372121979f65523d9daab7970d7ae461801bddda83d8"
Oct 30 19:16:24.806443 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:16:24.806415    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7a60343f435000d1f0f372121979f65523d9daab7970d7ae461801bddda83d8\": container with ID starting with b7a60343f435000d1f0f372121979f65523d9daab7970d7ae461801bddda83d8 not found: ID does not exist" containerID="b7a60343f435000d1f0f372121979f65523d9daab7970d7ae461801bddda83d8"
Oct 30 19:16:24.806722 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.806590    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7a60343f435000d1f0f372121979f65523d9daab7970d7ae461801bddda83d8"} err="failed to get container status \"b7a60343f435000d1f0f372121979f65523d9daab7970d7ae461801bddda83d8\": rpc error: code = NotFound desc = could not find container \"b7a60343f435000d1f0f372121979f65523d9daab7970d7ae461801bddda83d8\": container with ID starting with b7a60343f435000d1f0f372121979f65523d9daab7970d7ae461801bddda83d8 not found: ID does not exist"
Oct 30 19:16:24.952778 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.952695    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"]
Oct 30 19:16:24.957384 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:24.957359    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-5f80ac-566f66987d-rn5wk"]
Oct 30 19:16:26.602368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:26.602336    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fbd9d51e-133f-4b3a-b592-38ca4b2b5d06" path="/var/lib/kubelet/pods/fbd9d51e-133f-4b3a-b592-38ca4b2b5d06/volumes"
Oct 30 19:16:27.810174 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:27.810142    4544 generic.go:356] "Generic (PLEG): container finished" podID="34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" containerID="9bd2c6d9961d184842235dfcc10cea15730d9af3adb6ebfcb330e0ceafcba44f" exitCode=0
Oct 30 19:16:27.810531 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:27.810215    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg" event={"ID":"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd","Type":"ContainerDied","Data":"9bd2c6d9961d184842235dfcc10cea15730d9af3adb6ebfcb330e0ceafcba44f"}
Oct 30 19:16:27.912746 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:27.912716    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 19:16:28.034872 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.034834    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-kube-api-access-zz68p\" (UniqueName: \"kubernetes.io/projected/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-kube-api-access-zz68p\") pod \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") "
Oct 30 19:16:28.035071 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.034919    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-podinfo\") pod \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") "
Oct 30 19:16:28.035071 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.034960    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-credential-socket\") pod \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") "
Oct 30 19:16:28.035071 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.035009    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-workload-socket\") pod \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") "
Oct 30 19:16:28.035071 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.035063    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-data\") pod \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") "
Oct 30 19:16:28.035276 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.035109    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-envoy\") pod \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") "
Oct 30 19:16:28.035276 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.035232    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-token\" (UniqueName: \"kubernetes.io/projected/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-token\") pod \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") "
Oct 30 19:16:28.035356 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.035316    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istiod-ca-cert\") pod \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") "
Oct 30 19:16:28.035402 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.035369    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-workload-certs\") pod \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\" (UID: \"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd\") "
Oct 30 19:16:28.035453 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.035431    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-workload-socket" pod "34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" (UID: "34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:28.036255 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.035656    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-credential-socket" pod "34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" (UID: "34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:28.036255 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.035879    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-workload-certs" pod "34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" (UID: "34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:28.036255 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.035881    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:28.036255 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.035922    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:28.036255 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.035998    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-data" pod "34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" (UID: "34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:28.036255 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.036187    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istiod-ca-cert" pod "34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" (UID: "34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:28.037276 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.037251    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-podinfo" pod "34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" (UID: "34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:16:28.037515 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.037490    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-kube-api-access-zz68p" pod "34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" (UID: "34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd"). InnerVolumeSpecName "kube-api-access-zz68p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:28.038092 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.038067    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-token" pod "34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" (UID: "34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:28.038263 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.038241    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-envoy" pod "34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" (UID: "34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:28.137641 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.137082    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:28.137641 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.137118    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:28.137641 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.137133    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:28.137641 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.137147    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:28.137641 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.137160    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:28.137641 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.137177    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zz68p\" (UniqueName: \"kubernetes.io/projected/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-kube-api-access-zz68p\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:28.137641 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.137191    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:28.822126 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.822086    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg" event={"ID":"34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd","Type":"ContainerDied","Data":"72cda0569f9ea4db4b274c7027a46e9f9d08fc0949b0b9203ebb5baf0f2d0563"}
Oct 30 19:16:28.822556 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.822172    4544 scope.go:122] "RemoveContainer" containerID="9bd2c6d9961d184842235dfcc10cea15730d9af3adb6ebfcb330e0ceafcba44f"
Oct 30 19:16:28.822556 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.822108    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"
Oct 30 19:16:28.986032 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:28.985956    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"]
Oct 30 19:16:29.033076 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:29.033046    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-649d16-openshift-default-dbf76bc49-tf8xg"]
Oct 30 19:16:29.749275 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:29.749240    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"]
Oct 30 19:16:29.754436 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:29.754408    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"]
Oct 30 19:16:30.602218 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:30.602186    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" path="/var/lib/kubelet/pods/34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd/volumes"
Oct 30 19:16:35.531401 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:16:35.531362    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:16:42.931257 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:42.931230    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 19:16:42.934918 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:42.934894    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 19:16:44.770082 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:44.770031    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main" containerID="cri-o://65ed675a154c00dc393e32f838aed409a225c7d1175dc8d8f7684f27654de8f3" gracePeriod=45
Oct 30 19:16:44.774158 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:44.774128    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" containerID="cri-o://838fdf2a3794916d3a6aaa58be7f6f9de4d3ce49af14069a5843ebaecb1d37d8" gracePeriod=45
Oct 30 19:16:44.887906 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:44.887868    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main" probeResult="failure" output="Get \"https://10.133.0.61:8000/ready\": dial tcp 10.133.0.61:8000: connect: connection refused"
Oct 30 19:16:44.921636 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:44.921580    4544 generic.go:356] "Generic (PLEG): container finished" podID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerID="838fdf2a3794916d3a6aaa58be7f6f9de4d3ce49af14069a5843ebaecb1d37d8" exitCode=0
Oct 30 19:16:44.922128 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:44.921872    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" event={"ID":"bfb6e108-03a3-4d0f-94d4-2a29eee0d582","Type":"ContainerDied","Data":"838fdf2a3794916d3a6aaa58be7f6f9de4d3ce49af14069a5843ebaecb1d37d8"}
Oct 30 19:16:44.924241 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:44.924173    4544 generic.go:356] "Generic (PLEG): container finished" podID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerID="65ed675a154c00dc393e32f838aed409a225c7d1175dc8d8f7684f27654de8f3" exitCode=0
Oct 30 19:16:44.924241 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:44.924204    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" event={"ID":"fd9777c9-1b40-42fd-9815-cdf1a3509661","Type":"ContainerDied","Data":"65ed675a154c00dc393e32f838aed409a225c7d1175dc8d8f7684f27654de8f3"}
Oct 30 19:16:45.077136 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.077103    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 19:16:45.081387 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.081363    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 19:16:45.140333 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.140297    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-tmp-dir\") pod \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") "
Oct 30 19:16:45.140493 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.140348    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd9777c9-1b40-42fd-9815-cdf1a3509661-kube-api-access-4dqvv\" (UniqueName: \"kubernetes.io/projected/fd9777c9-1b40-42fd-9815-cdf1a3509661-kube-api-access-4dqvv\") pod \"fd9777c9-1b40-42fd-9815-cdf1a3509661\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") "
Oct 30 19:16:45.140493 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.140399    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-dshm\") pod \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") "
Oct 30 19:16:45.140493 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.140427    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-dshm\") pod \"fd9777c9-1b40-42fd-9815-cdf1a3509661\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") "
Oct 30 19:16:45.140493 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.140480    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-home\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-home\") pod \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") "
Oct 30 19:16:45.140743 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.140516    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-model-cache\") pod \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") "
Oct 30 19:16:45.140743 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.140519    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-tmp-dir" pod "bfb6e108-03a3-4d0f-94d4-2a29eee0d582" (UID: "bfb6e108-03a3-4d0f-94d4-2a29eee0d582"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:45.140743 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.140549    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-kube-api-access-ngnnj\" (UniqueName: \"kubernetes.io/projected/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-kube-api-access-ngnnj\") pod \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") "
Oct 30 19:16:45.140743 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.140589    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-home\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-home\") pod \"fd9777c9-1b40-42fd-9815-cdf1a3509661\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") "
Oct 30 19:16:45.140743 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.140699    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-tls-certs\" (UniqueName: \"kubernetes.io/secret/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-tls-certs\") pod \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\" (UID: \"bfb6e108-03a3-4d0f-94d4-2a29eee0d582\") "
Oct 30 19:16:45.140743 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.140744    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-tmp-dir\") pod \"fd9777c9-1b40-42fd-9815-cdf1a3509661\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") "
Oct 30 19:16:45.141057 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.140775    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fd9777c9-1b40-42fd-9815-cdf1a3509661-tls-certs\" (UniqueName: \"kubernetes.io/secret/fd9777c9-1b40-42fd-9815-cdf1a3509661-tls-certs\") pod \"fd9777c9-1b40-42fd-9815-cdf1a3509661\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") "
Oct 30 19:16:45.141057 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.140851    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-model-cache\") pod \"fd9777c9-1b40-42fd-9815-cdf1a3509661\" (UID: \"fd9777c9-1b40-42fd-9815-cdf1a3509661\") "
Oct 30 19:16:45.141057 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.140861    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-model-cache" pod "bfb6e108-03a3-4d0f-94d4-2a29eee0d582" (UID: "bfb6e108-03a3-4d0f-94d4-2a29eee0d582"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:45.141279 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.141056    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-home" pod "fd9777c9-1b40-42fd-9815-cdf1a3509661" (UID: "fd9777c9-1b40-42fd-9815-cdf1a3509661"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:45.141279 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.141264    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-home" pod "bfb6e108-03a3-4d0f-94d4-2a29eee0d582" (UID: "bfb6e108-03a3-4d0f-94d4-2a29eee0d582"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:45.141397 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.141271    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-tmp-dir" pod "fd9777c9-1b40-42fd-9815-cdf1a3509661" (UID: "fd9777c9-1b40-42fd-9815-cdf1a3509661"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:45.141397 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.141386    4544 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-tmp-dir\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:45.141517 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.141407    4544 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-home\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:45.141517 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.141421    4544 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-model-cache\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:45.141517 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.141434    4544 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-home\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:45.141517 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.141450    4544 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-tmp-dir\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:45.141753 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.141591    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-model-cache" pod "fd9777c9-1b40-42fd-9815-cdf1a3509661" (UID: "fd9777c9-1b40-42fd-9815-cdf1a3509661"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:45.143265 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.143226    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-dshm" pod "fd9777c9-1b40-42fd-9815-cdf1a3509661" (UID: "fd9777c9-1b40-42fd-9815-cdf1a3509661"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:45.143391 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.143340    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-dshm" pod "bfb6e108-03a3-4d0f-94d4-2a29eee0d582" (UID: "bfb6e108-03a3-4d0f-94d4-2a29eee0d582"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:45.143724 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.143699    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd9777c9-1b40-42fd-9815-cdf1a3509661-kube-api-access-4dqvv" pod "fd9777c9-1b40-42fd-9815-cdf1a3509661" (UID: "fd9777c9-1b40-42fd-9815-cdf1a3509661"). InnerVolumeSpecName "kube-api-access-4dqvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:45.143821 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.143715    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-tls-certs" pod "bfb6e108-03a3-4d0f-94d4-2a29eee0d582" (UID: "bfb6e108-03a3-4d0f-94d4-2a29eee0d582"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:45.143821 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.143707    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd9777c9-1b40-42fd-9815-cdf1a3509661-tls-certs" pod "fd9777c9-1b40-42fd-9815-cdf1a3509661" (UID: "fd9777c9-1b40-42fd-9815-cdf1a3509661"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:45.143821 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.143753    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-kube-api-access-ngnnj" pod "bfb6e108-03a3-4d0f-94d4-2a29eee0d582" (UID: "bfb6e108-03a3-4d0f-94d4-2a29eee0d582"). InnerVolumeSpecName "kube-api-access-ngnnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:45.242629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.242525    4544 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-tls-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:45.242629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.242552    4544 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd9777c9-1b40-42fd-9815-cdf1a3509661-tls-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:45.242629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.242562    4544 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-model-cache\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:45.242629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.242574    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dqvv\" (UniqueName: \"kubernetes.io/projected/fd9777c9-1b40-42fd-9815-cdf1a3509661-kube-api-access-4dqvv\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:45.242629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.242590    4544 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-dshm\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:45.242629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.242614    4544 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd9777c9-1b40-42fd-9815-cdf1a3509661-dshm\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:45.242629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.242624    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngnnj\" (UniqueName: \"kubernetes.io/projected/bfb6e108-03a3-4d0f-94d4-2a29eee0d582-kube-api-access-ngnnj\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:45.931755 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.931721    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"
Oct 30 19:16:45.931755 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.931738    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw" event={"ID":"bfb6e108-03a3-4d0f-94d4-2a29eee0d582","Type":"ContainerDied","Data":"d2a623b25f0269edc2c9939819667598511c016e9d9f092a0de3fd7f91e5c37c"}
Oct 30 19:16:45.932287 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.931788    4544 scope.go:122] "RemoveContainer" containerID="838fdf2a3794916d3a6aaa58be7f6f9de4d3ce49af14069a5843ebaecb1d37d8"
Oct 30 19:16:45.933329 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.933299    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v" event={"ID":"fd9777c9-1b40-42fd-9815-cdf1a3509661","Type":"ContainerDied","Data":"fa6904fbbdb956b740ff19adbf5aaaafbe84ffada89362d45231afeb8cf44e90"}
Oct 30 19:16:45.933440 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.933315    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"
Oct 30 19:16:45.942758 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:45.942736    4544 scope.go:122] "RemoveContainer" containerID="65ed675a154c00dc393e32f838aed409a225c7d1175dc8d8f7684f27654de8f3"
Oct 30 19:16:46.088250 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:46.088157    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"]
Oct 30 19:16:46.133764 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:46.133724    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-75e91b/distinct-2-w4-75e91b-kserve-7897fdd4f5-qcw7v"]
Oct 30 19:16:46.239045 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:46.239010    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"]
Oct 30 19:16:46.242902 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:46.242875    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-75e91b/distinct-w4-75e91b-kserve-595cf9cb76-l6lcw"]
Oct 30 19:16:46.439771 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:46.439693    4544 patch_prober.go:29] interesting pod/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:16:46.439771 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:46.439741    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8" podUID="90685ad5-9312-489e-ab58-84c12ebcc5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:16:46.602826 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:46.602795    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" path="/var/lib/kubelet/pods/bfb6e108-03a3-4d0f-94d4-2a29eee0d582/volumes"
Oct 30 19:16:46.603346 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:46.603325    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" path="/var/lib/kubelet/pods/fd9777c9-1b40-42fd-9815-cdf1a3509661/volumes"
Oct 30 19:16:48.953439 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:48.953404    4544 generic.go:356] "Generic (PLEG): container finished" podID="90685ad5-9312-489e-ab58-84c12ebcc5ab" containerID="1541c711d30aa1903b3b2c837e2cf0a78a2402858490f4f01a9c25dd97ea34ec" exitCode=0
Oct 30 19:16:48.953903 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:48.953439    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8" event={"ID":"90685ad5-9312-489e-ab58-84c12ebcc5ab","Type":"ContainerDied","Data":"1541c711d30aa1903b3b2c837e2cf0a78a2402858490f4f01a9c25dd97ea34ec"}
Oct 30 19:16:48.953903 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:48.953878    4544 scope.go:122] "RemoveContainer" containerID="1541c711d30aa1903b3b2c837e2cf0a78a2402858490f4f01a9c25dd97ea34ec"
Oct 30 19:16:48.955214 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:48.955185    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:16:48.955326 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:48.955268    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:16:48.955326 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:48.955314    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:16:49.984688 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:49.984654    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8" event={"ID":"90685ad5-9312-489e-ab58-84c12ebcc5ab","Type":"ContainerStarted","Data":"c88eb83111408120dbbe38b4f735285b0cead33afc72e1c5248298739a44ed37"}
Oct 30 19:16:50.845022 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:50.844986    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:16:50.845252 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:50.845185    4544 patch_prober.go:29] interesting pod/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:16:50.845336 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:50.845255    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8" podUID="90685ad5-9312-489e-ab58-84c12ebcc5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Oct 30 19:16:51.846103 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:51.846071    4544 patch_prober.go:29] interesting pod/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:16:51.846465 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:51.846140    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8" podUID="90685ad5-9312-489e-ab58-84c12ebcc5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Oct 30 19:16:52.845467 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:52.845434    4544 patch_prober.go:29] interesting pod/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:16:52.845675 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:52.845488    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8" podUID="90685ad5-9312-489e-ab58-84c12ebcc5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Oct 30 19:16:53.845958 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:53.845919    4544 patch_prober.go:29] interesting pod/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:16:53.846361 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:53.845979    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8" podUID="90685ad5-9312-489e-ab58-84c12ebcc5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Oct 30 19:16:54.845381 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:54.845345    4544 patch_prober.go:29] interesting pod/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:16:54.845571 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:54.845397    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8" podUID="90685ad5-9312-489e-ab58-84c12ebcc5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Oct 30 19:16:55.849492 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:55.849461    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:16:55.849961 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:55.849808    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:16:55.850558 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:16:55.850541    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:17:04.357779 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:04.357745    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"]
Oct 30 19:17:04.358307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:04.357992    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj" podUID="6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" containerName="istio-proxy" containerID="cri-o://03ba2e5b25ef574211f41d1aa8171ebf7d7a3aaa28d70885b9288298a6aab32e" gracePeriod=30
Oct 30 19:17:04.373622 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:04.373577    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"]
Oct 30 19:17:04.373805 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:04.373786    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc" podUID="e4dc6f73-5287-4f71-b368-6c0d605e9054" containerName="istio-proxy" containerID="cri-o://5eed08e9680e8ca3d075b8099176687bfbc007b0317d12f759a3a42fb37e5a4c" gracePeriod=30
Oct 30 19:17:04.387319 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:04.387241    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"]
Oct 30 19:17:04.387590 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:04.387548    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4" podUID="14497a85-b284-431d-930f-a0ac9274ab6c" containerName="istio-proxy" containerID="cri-o://9cf1e5617bc9ad999c364179612169a3b9e2f5d93b72254a64486e30690a974f" gracePeriod=30
Oct 30 19:17:05.537214 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:17:05.537171    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:17:09.664325 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.664294    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:17:09.692340 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.692317    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:17:09.696417 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.696399    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:17:09.712429 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.712407    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-podinfo\") pod \"e4dc6f73-5287-4f71-b368-6c0d605e9054\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") "
Oct 30 19:17:09.712547 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.712449    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-workload-socket\") pod \"e4dc6f73-5287-4f71-b368-6c0d605e9054\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") "
Oct 30 19:17:09.712547 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.712487    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4dc6f73-5287-4f71-b368-6c0d605e9054-kube-api-access-dhvw6\" (UniqueName: \"kubernetes.io/projected/e4dc6f73-5287-4f71-b368-6c0d605e9054-kube-api-access-dhvw6\") pod \"e4dc6f73-5287-4f71-b368-6c0d605e9054\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") "
Oct 30 19:17:09.712547 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.712519    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-workload-certs\") pod \"e4dc6f73-5287-4f71-b368-6c0d605e9054\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") "
Oct 30 19:17:09.712738 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.712594    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-envoy\") pod \"e4dc6f73-5287-4f71-b368-6c0d605e9054\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") "
Oct 30 19:17:09.712738 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.712704    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-token\" (UniqueName: \"kubernetes.io/projected/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-token\") pod \"e4dc6f73-5287-4f71-b368-6c0d605e9054\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") "
Oct 30 19:17:09.712836 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.712742    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-credential-socket\") pod \"e4dc6f73-5287-4f71-b368-6c0d605e9054\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") "
Oct 30 19:17:09.712836 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.712809    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-workload-socket" pod "e4dc6f73-5287-4f71-b368-6c0d605e9054" (UID: "e4dc6f73-5287-4f71-b368-6c0d605e9054"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.712836 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.712829    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-data\") pod \"e4dc6f73-5287-4f71-b368-6c0d605e9054\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") "
Oct 30 19:17:09.712991 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.712905    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e4dc6f73-5287-4f71-b368-6c0d605e9054-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4dc6f73-5287-4f71-b368-6c0d605e9054-istiod-ca-cert\") pod \"e4dc6f73-5287-4f71-b368-6c0d605e9054\" (UID: \"e4dc6f73-5287-4f71-b368-6c0d605e9054\") "
Oct 30 19:17:09.713043 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.713021    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-workload-certs" pod "e4dc6f73-5287-4f71-b368-6c0d605e9054" (UID: "e4dc6f73-5287-4f71-b368-6c0d605e9054"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.713096 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.713036    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-credential-socket" pod "e4dc6f73-5287-4f71-b368-6c0d605e9054" (UID: "e4dc6f73-5287-4f71-b368-6c0d605e9054"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.713255 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.713233    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-data" pod "e4dc6f73-5287-4f71-b368-6c0d605e9054" (UID: "e4dc6f73-5287-4f71-b368-6c0d605e9054"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.713439 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.713419    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.713514 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.713444    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.713514 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.713441    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4dc6f73-5287-4f71-b368-6c0d605e9054-istiod-ca-cert" pod "e4dc6f73-5287-4f71-b368-6c0d605e9054" (UID: "e4dc6f73-5287-4f71-b368-6c0d605e9054"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:09.713514 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.713478    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.713514 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.713501    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.714733 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.714711    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-podinfo" pod "e4dc6f73-5287-4f71-b368-6c0d605e9054" (UID: "e4dc6f73-5287-4f71-b368-6c0d605e9054"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:09.714930 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.714903    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-token" pod "e4dc6f73-5287-4f71-b368-6c0d605e9054" (UID: "e4dc6f73-5287-4f71-b368-6c0d605e9054"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:09.714974 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.714917    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4dc6f73-5287-4f71-b368-6c0d605e9054-kube-api-access-dhvw6" pod "e4dc6f73-5287-4f71-b368-6c0d605e9054" (UID: "e4dc6f73-5287-4f71-b368-6c0d605e9054"). InnerVolumeSpecName "kube-api-access-dhvw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:09.715904 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.715884    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-envoy" pod "e4dc6f73-5287-4f71-b368-6c0d605e9054" (UID: "e4dc6f73-5287-4f71-b368-6c0d605e9054"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.813950 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.813914    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istiod-ca-cert\") pod \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") "
Oct 30 19:17:09.814177 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.813964    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-workload-certs\") pod \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") "
Oct 30 19:17:09.814177 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.814012    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-kube-api-access-psn5p\" (UniqueName: \"kubernetes.io/projected/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-kube-api-access-psn5p\") pod \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") "
Oct 30 19:17:09.814177 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.814048    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-envoy\") pod \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") "
Oct 30 19:17:09.814177 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.814084    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-credential-socket\") pod \"14497a85-b284-431d-930f-a0ac9274ab6c\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") "
Oct 30 19:17:09.814177 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.814127    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-istio-data\") pod \"14497a85-b284-431d-930f-a0ac9274ab6c\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") "
Oct 30 19:17:09.814483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.814186    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14497a85-b284-431d-930f-a0ac9274ab6c-kube-api-access-66ccd\" (UniqueName: \"kubernetes.io/projected/14497a85-b284-431d-930f-a0ac9274ab6c-kube-api-access-66ccd\") pod \"14497a85-b284-431d-930f-a0ac9274ab6c\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") "
Oct 30 19:17:09.814483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.814216    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-podinfo\") pod \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") "
Oct 30 19:17:09.814483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.814298    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14497a85-b284-431d-930f-a0ac9274ab6c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14497a85-b284-431d-930f-a0ac9274ab6c-istiod-ca-cert\") pod \"14497a85-b284-431d-930f-a0ac9274ab6c\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") "
Oct 30 19:17:09.814483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.814331    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-workload-socket\") pod \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") "
Oct 30 19:17:09.814483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.814374    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-workload-socket\") pod \"14497a85-b284-431d-930f-a0ac9274ab6c\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") "
Oct 30 19:17:09.814483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.814407    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/14497a85-b284-431d-930f-a0ac9274ab6c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14497a85-b284-431d-930f-a0ac9274ab6c-istio-podinfo\") pod \"14497a85-b284-431d-930f-a0ac9274ab6c\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") "
Oct 30 19:17:09.814483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.814425    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-istio-envoy\") pod \"14497a85-b284-431d-930f-a0ac9274ab6c\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") "
Oct 30 19:17:09.814885 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.814489    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-data\") pod \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") "
Oct 30 19:17:09.814885 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.814526    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-workload-certs\") pod \"14497a85-b284-431d-930f-a0ac9274ab6c\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") "
Oct 30 19:17:09.814885 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.814553    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14497a85-b284-431d-930f-a0ac9274ab6c-istio-token\" (UniqueName: \"kubernetes.io/projected/14497a85-b284-431d-930f-a0ac9274ab6c-istio-token\") pod \"14497a85-b284-431d-930f-a0ac9274ab6c\" (UID: \"14497a85-b284-431d-930f-a0ac9274ab6c\") "
Oct 30 19:17:09.814885 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.814582    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-credential-socket\") pod \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") "
Oct 30 19:17:09.814885 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.814658    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-token\" (UniqueName: \"kubernetes.io/projected/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-token\") pod \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\" (UID: \"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95\") "
Oct 30 19:17:09.814885 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.814677    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-workload-certs" pod "6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" (UID: "6276b3df-6da0-43fa-8bf1-2ffa76f0ce95"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.815188 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.814975    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istiod-ca-cert" pod "6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" (UID: "6276b3df-6da0-43fa-8bf1-2ffa76f0ce95"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:09.815188 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.815126    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4dc6f73-5287-4f71-b368-6c0d605e9054-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.815188 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.815156    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.815188 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.815172    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.815188 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.815185    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.815443 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.815200    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhvw6\" (UniqueName: \"kubernetes.io/projected/e4dc6f73-5287-4f71-b368-6c0d605e9054-kube-api-access-dhvw6\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.815443 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.815217    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.815443 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.815230    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4dc6f73-5287-4f71-b368-6c0d605e9054-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.815443 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.815303    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-workload-socket" pod "14497a85-b284-431d-930f-a0ac9274ab6c" (UID: "14497a85-b284-431d-930f-a0ac9274ab6c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.815729 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.815534    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-data" pod "6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" (UID: "6276b3df-6da0-43fa-8bf1-2ffa76f0ce95"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.816318 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.815792    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-credential-socket" pod "14497a85-b284-431d-930f-a0ac9274ab6c" (UID: "14497a85-b284-431d-930f-a0ac9274ab6c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.816318 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.816017    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-istio-data" pod "14497a85-b284-431d-930f-a0ac9274ab6c" (UID: "14497a85-b284-431d-930f-a0ac9274ab6c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.816318 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.816037    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-workload-certs" pod "14497a85-b284-431d-930f-a0ac9274ab6c" (UID: "14497a85-b284-431d-930f-a0ac9274ab6c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.816318 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.816253    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-credential-socket" pod "6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" (UID: "6276b3df-6da0-43fa-8bf1-2ffa76f0ce95"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.817262 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.817234    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14497a85-b284-431d-930f-a0ac9274ab6c-istiod-ca-cert" pod "14497a85-b284-431d-930f-a0ac9274ab6c" (UID: "14497a85-b284-431d-930f-a0ac9274ab6c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:09.817376 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.817355    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-workload-socket" pod "6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" (UID: "6276b3df-6da0-43fa-8bf1-2ffa76f0ce95"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.818110 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.818084    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-token" pod "6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" (UID: "6276b3df-6da0-43fa-8bf1-2ffa76f0ce95"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:09.818711 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.818678    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-istio-envoy" pod "14497a85-b284-431d-930f-a0ac9274ab6c" (UID: "14497a85-b284-431d-930f-a0ac9274ab6c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.818898 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.818874    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14497a85-b284-431d-930f-a0ac9274ab6c-kube-api-access-66ccd" pod "14497a85-b284-431d-930f-a0ac9274ab6c" (UID: "14497a85-b284-431d-930f-a0ac9274ab6c"). InnerVolumeSpecName "kube-api-access-66ccd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:09.819332 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.819304    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14497a85-b284-431d-930f-a0ac9274ab6c-istio-token" pod "14497a85-b284-431d-930f-a0ac9274ab6c" (UID: "14497a85-b284-431d-930f-a0ac9274ab6c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:09.819435 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.819312    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-envoy" pod "6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" (UID: "6276b3df-6da0-43fa-8bf1-2ffa76f0ce95"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.819741 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.819717    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-podinfo" pod "6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" (UID: "6276b3df-6da0-43fa-8bf1-2ffa76f0ce95"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:09.820268 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.820237    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/14497a85-b284-431d-930f-a0ac9274ab6c-istio-podinfo" pod "14497a85-b284-431d-930f-a0ac9274ab6c" (UID: "14497a85-b284-431d-930f-a0ac9274ab6c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:09.820625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.820579    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-kube-api-access-psn5p" pod "6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" (UID: "6276b3df-6da0-43fa-8bf1-2ffa76f0ce95"). InnerVolumeSpecName "kube-api-access-psn5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:09.916872 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.916794    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.916872 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.916826    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14497a85-b284-431d-930f-a0ac9274ab6c-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.916872 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.916842    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.916872 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.916858    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.916872 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.916873    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.917193 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.916889    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14497a85-b284-431d-930f-a0ac9274ab6c-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.917193 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.916903    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.917193 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.916918    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.917193 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.916932    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psn5p\" (UniqueName: \"kubernetes.io/projected/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-kube-api-access-psn5p\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.917193 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.916947    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.917193 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.916962    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.917193 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.916977    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14497a85-b284-431d-930f-a0ac9274ab6c-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.917193 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.916991    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66ccd\" (UniqueName: \"kubernetes.io/projected/14497a85-b284-431d-930f-a0ac9274ab6c-kube-api-access-66ccd\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.917193 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.917005    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.917193 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.917020    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14497a85-b284-431d-930f-a0ac9274ab6c-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.917193 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:09.917033    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:10.106335 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.106299    4544 generic.go:356] "Generic (PLEG): container finished" podID="6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" containerID="03ba2e5b25ef574211f41d1aa8171ebf7d7a3aaa28d70885b9288298a6aab32e" exitCode=0
Oct 30 19:17:10.106574 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.106358    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"
Oct 30 19:17:10.106574 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.106391    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj" event={"ID":"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95","Type":"ContainerDied","Data":"03ba2e5b25ef574211f41d1aa8171ebf7d7a3aaa28d70885b9288298a6aab32e"}
Oct 30 19:17:10.106574 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.106458    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj" event={"ID":"6276b3df-6da0-43fa-8bf1-2ffa76f0ce95","Type":"ContainerDied","Data":"3ef897813bff3f1115797c52b3265261d457d486a59b110739f971ba37e2cc61"}
Oct 30 19:17:10.106574 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.106487    4544 scope.go:122] "RemoveContainer" containerID="03ba2e5b25ef574211f41d1aa8171ebf7d7a3aaa28d70885b9288298a6aab32e"
Oct 30 19:17:10.108840 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.108815    4544 generic.go:356] "Generic (PLEG): container finished" podID="14497a85-b284-431d-930f-a0ac9274ab6c" containerID="9cf1e5617bc9ad999c364179612169a3b9e2f5d93b72254a64486e30690a974f" exitCode=0
Oct 30 19:17:10.108993 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.108881    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"
Oct 30 19:17:10.108993 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.108942    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4" event={"ID":"14497a85-b284-431d-930f-a0ac9274ab6c","Type":"ContainerDied","Data":"9cf1e5617bc9ad999c364179612169a3b9e2f5d93b72254a64486e30690a974f"}
Oct 30 19:17:10.108993 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.108982    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4" event={"ID":"14497a85-b284-431d-930f-a0ac9274ab6c","Type":"ContainerDied","Data":"24ee10bd7c5cdefe065c6f810da5aa5dce3c4143c0a3389d996381034ce0ab54"}
Oct 30 19:17:10.118167 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.118140    4544 generic.go:356] "Generic (PLEG): container finished" podID="e4dc6f73-5287-4f71-b368-6c0d605e9054" containerID="5eed08e9680e8ca3d075b8099176687bfbc007b0317d12f759a3a42fb37e5a4c" exitCode=0
Oct 30 19:17:10.118280 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.118199    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc" event={"ID":"e4dc6f73-5287-4f71-b368-6c0d605e9054","Type":"ContainerDied","Data":"5eed08e9680e8ca3d075b8099176687bfbc007b0317d12f759a3a42fb37e5a4c"}
Oct 30 19:17:10.118280 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.118242    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc" event={"ID":"e4dc6f73-5287-4f71-b368-6c0d605e9054","Type":"ContainerDied","Data":"1c43718f8ef73cb842a15c2cbaa0ca2a45d4141912477cbb88d747d0d078f06a"}
Oct 30 19:17:10.118401 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.118290    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"
Oct 30 19:17:10.118526 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.118430    4544 scope.go:122] "RemoveContainer" containerID="03ba2e5b25ef574211f41d1aa8171ebf7d7a3aaa28d70885b9288298a6aab32e"
Oct 30 19:17:10.118745 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:17:10.118727    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03ba2e5b25ef574211f41d1aa8171ebf7d7a3aaa28d70885b9288298a6aab32e\": container with ID starting with 03ba2e5b25ef574211f41d1aa8171ebf7d7a3aaa28d70885b9288298a6aab32e not found: ID does not exist" containerID="03ba2e5b25ef574211f41d1aa8171ebf7d7a3aaa28d70885b9288298a6aab32e"
Oct 30 19:17:10.118839 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.118753    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03ba2e5b25ef574211f41d1aa8171ebf7d7a3aaa28d70885b9288298a6aab32e"} err="failed to get container status \"03ba2e5b25ef574211f41d1aa8171ebf7d7a3aaa28d70885b9288298a6aab32e\": rpc error: code = NotFound desc = could not find container \"03ba2e5b25ef574211f41d1aa8171ebf7d7a3aaa28d70885b9288298a6aab32e\": container with ID starting with 03ba2e5b25ef574211f41d1aa8171ebf7d7a3aaa28d70885b9288298a6aab32e not found: ID does not exist"
Oct 30 19:17:10.118839 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.118768    4544 scope.go:122] "RemoveContainer" containerID="9cf1e5617bc9ad999c364179612169a3b9e2f5d93b72254a64486e30690a974f"
Oct 30 19:17:10.129687 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.129591    4544 scope.go:122] "RemoveContainer" containerID="5450673dbd1bdb87f3cdc593241e287b3e8243744a944de9c5e594cb18f1e157"
Oct 30 19:17:10.138070 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.138052    4544 scope.go:122] "RemoveContainer" containerID="9cf1e5617bc9ad999c364179612169a3b9e2f5d93b72254a64486e30690a974f"
Oct 30 19:17:10.138352 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:17:10.138323    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cf1e5617bc9ad999c364179612169a3b9e2f5d93b72254a64486e30690a974f\": container with ID starting with 9cf1e5617bc9ad999c364179612169a3b9e2f5d93b72254a64486e30690a974f not found: ID does not exist" containerID="9cf1e5617bc9ad999c364179612169a3b9e2f5d93b72254a64486e30690a974f"
Oct 30 19:17:10.138445 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.138357    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cf1e5617bc9ad999c364179612169a3b9e2f5d93b72254a64486e30690a974f"} err="failed to get container status \"9cf1e5617bc9ad999c364179612169a3b9e2f5d93b72254a64486e30690a974f\": rpc error: code = NotFound desc = could not find container \"9cf1e5617bc9ad999c364179612169a3b9e2f5d93b72254a64486e30690a974f\": container with ID starting with 9cf1e5617bc9ad999c364179612169a3b9e2f5d93b72254a64486e30690a974f not found: ID does not exist"
Oct 30 19:17:10.138445 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.138377    4544 scope.go:122] "RemoveContainer" containerID="5450673dbd1bdb87f3cdc593241e287b3e8243744a944de9c5e594cb18f1e157"
Oct 30 19:17:10.138710 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:17:10.138688    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5450673dbd1bdb87f3cdc593241e287b3e8243744a944de9c5e594cb18f1e157\": container with ID starting with 5450673dbd1bdb87f3cdc593241e287b3e8243744a944de9c5e594cb18f1e157 not found: ID does not exist" containerID="5450673dbd1bdb87f3cdc593241e287b3e8243744a944de9c5e594cb18f1e157"
Oct 30 19:17:10.138825 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.138721    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5450673dbd1bdb87f3cdc593241e287b3e8243744a944de9c5e594cb18f1e157"} err="failed to get container status \"5450673dbd1bdb87f3cdc593241e287b3e8243744a944de9c5e594cb18f1e157\": rpc error: code = NotFound desc = could not find container \"5450673dbd1bdb87f3cdc593241e287b3e8243744a944de9c5e594cb18f1e157\": container with ID starting with 5450673dbd1bdb87f3cdc593241e287b3e8243744a944de9c5e594cb18f1e157 not found: ID does not exist"
Oct 30 19:17:10.138825 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.138740    4544 scope.go:122] "RemoveContainer" containerID="5eed08e9680e8ca3d075b8099176687bfbc007b0317d12f759a3a42fb37e5a4c"
Oct 30 19:17:10.146658 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.146633    4544 scope.go:122] "RemoveContainer" containerID="5eed08e9680e8ca3d075b8099176687bfbc007b0317d12f759a3a42fb37e5a4c"
Oct 30 19:17:10.146972 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:17:10.146949    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eed08e9680e8ca3d075b8099176687bfbc007b0317d12f759a3a42fb37e5a4c\": container with ID starting with 5eed08e9680e8ca3d075b8099176687bfbc007b0317d12f759a3a42fb37e5a4c not found: ID does not exist" containerID="5eed08e9680e8ca3d075b8099176687bfbc007b0317d12f759a3a42fb37e5a4c"
Oct 30 19:17:10.147046 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.146982    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eed08e9680e8ca3d075b8099176687bfbc007b0317d12f759a3a42fb37e5a4c"} err="failed to get container status \"5eed08e9680e8ca3d075b8099176687bfbc007b0317d12f759a3a42fb37e5a4c\": rpc error: code = NotFound desc = could not find container \"5eed08e9680e8ca3d075b8099176687bfbc007b0317d12f759a3a42fb37e5a4c\": container with ID starting with 5eed08e9680e8ca3d075b8099176687bfbc007b0317d12f759a3a42fb37e5a4c not found: ID does not exist"
Oct 30 19:17:10.262572 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.262500    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"]
Oct 30 19:17:10.267133 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.267111    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-tzmqc"]
Oct 30 19:17:10.371867 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.371835    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"]
Oct 30 19:17:10.378095 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.378065    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-ddszj"]
Oct 30 19:17:10.482370 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.482335    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"]
Oct 30 19:17:10.487544 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.487509    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-314bd6fa-openshift-default-69d8f74b5c-sbhk4"]
Oct 30 19:17:10.601712 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.601687    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14497a85-b284-431d-930f-a0ac9274ab6c" path="/var/lib/kubelet/pods/14497a85-b284-431d-930f-a0ac9274ab6c/volumes"
Oct 30 19:17:10.602177 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.602164    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" path="/var/lib/kubelet/pods/6276b3df-6da0-43fa-8bf1-2ffa76f0ce95/volumes"
Oct 30 19:17:10.602559 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:10.602547    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e4dc6f73-5287-4f71-b368-6c0d605e9054" path="/var/lib/kubelet/pods/e4dc6f73-5287-4f71-b368-6c0d605e9054/volumes"
Oct 30 19:17:27.185169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:27.185130    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"]
Oct 30 19:17:27.231730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:27.231701    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"]
Oct 30 19:17:27.246188 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:27.246154    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"]
Oct 30 19:17:27.246355 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:27.246335    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq" podUID="e45c27e0-1404-48e3-8289-6426eee88141" containerName="maas-api" containerID="cri-o://76b0fe820ae60c4adbe3d4b6e0b3f7ed4c70088e4533c97deeb072363c8beef7" gracePeriod=30
Oct 30 19:17:27.498329 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:27.498305    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 19:17:27.601035 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:27.601007    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e45c27e0-1404-48e3-8289-6426eee88141-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45c27e0-1404-48e3-8289-6426eee88141-maas-api-tls\") pod \"e45c27e0-1404-48e3-8289-6426eee88141\" (UID: \"e45c27e0-1404-48e3-8289-6426eee88141\") "
Oct 30 19:17:27.601192 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:27.601074    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e45c27e0-1404-48e3-8289-6426eee88141-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45c27e0-1404-48e3-8289-6426eee88141-openshift-service-ca\") pod \"e45c27e0-1404-48e3-8289-6426eee88141\" (UID: \"e45c27e0-1404-48e3-8289-6426eee88141\") "
Oct 30 19:17:27.601192 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:27.601138    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e45c27e0-1404-48e3-8289-6426eee88141-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45c27e0-1404-48e3-8289-6426eee88141-metrics-certs\") pod \"e45c27e0-1404-48e3-8289-6426eee88141\" (UID: \"e45c27e0-1404-48e3-8289-6426eee88141\") "
Oct 30 19:17:27.601278 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:27.601193    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e45c27e0-1404-48e3-8289-6426eee88141-kube-api-access-dfg75\" (UniqueName: \"kubernetes.io/projected/e45c27e0-1404-48e3-8289-6426eee88141-kube-api-access-dfg75\") pod \"e45c27e0-1404-48e3-8289-6426eee88141\" (UID: \"e45c27e0-1404-48e3-8289-6426eee88141\") "
Oct 30 19:17:27.601495 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:27.601458    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e45c27e0-1404-48e3-8289-6426eee88141-openshift-service-ca" pod "e45c27e0-1404-48e3-8289-6426eee88141" (UID: "e45c27e0-1404-48e3-8289-6426eee88141"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:27.603131 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:27.603107    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e45c27e0-1404-48e3-8289-6426eee88141-metrics-certs" pod "e45c27e0-1404-48e3-8289-6426eee88141" (UID: "e45c27e0-1404-48e3-8289-6426eee88141"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:17:27.603245 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:27.603156    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e45c27e0-1404-48e3-8289-6426eee88141-maas-api-tls" pod "e45c27e0-1404-48e3-8289-6426eee88141" (UID: "e45c27e0-1404-48e3-8289-6426eee88141"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:17:27.603245 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:27.603226    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e45c27e0-1404-48e3-8289-6426eee88141-kube-api-access-dfg75" pod "e45c27e0-1404-48e3-8289-6426eee88141" (UID: "e45c27e0-1404-48e3-8289-6426eee88141"). InnerVolumeSpecName "kube-api-access-dfg75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:27.702136 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:27.702105    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfg75\" (UniqueName: \"kubernetes.io/projected/e45c27e0-1404-48e3-8289-6426eee88141-kube-api-access-dfg75\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:27.702136 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:27.702136    4544 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e45c27e0-1404-48e3-8289-6426eee88141-maas-api-tls\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:27.702339 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:27.702150    4544 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e45c27e0-1404-48e3-8289-6426eee88141-openshift-service-ca\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:27.702339 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:27.702159    4544 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e45c27e0-1404-48e3-8289-6426eee88141-metrics-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:28.217922 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:28.217882    4544 generic.go:356] "Generic (PLEG): container finished" podID="e45c27e0-1404-48e3-8289-6426eee88141" containerID="76b0fe820ae60c4adbe3d4b6e0b3f7ed4c70088e4533c97deeb072363c8beef7" exitCode=0
Oct 30 19:17:28.218362 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:28.217934    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"
Oct 30 19:17:28.218362 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:28.217932    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq" event={"ID":"e45c27e0-1404-48e3-8289-6426eee88141","Type":"ContainerDied","Data":"76b0fe820ae60c4adbe3d4b6e0b3f7ed4c70088e4533c97deeb072363c8beef7"}
Oct 30 19:17:28.218362 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:28.217997    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq" event={"ID":"e45c27e0-1404-48e3-8289-6426eee88141","Type":"ContainerDied","Data":"94c5be1c005636ee8d36b807ecb98f3af8589aea005fee474d39cf50c8269cf0"}
Oct 30 19:17:28.218362 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:28.218019    4544 scope.go:122] "RemoveContainer" containerID="76b0fe820ae60c4adbe3d4b6e0b3f7ed4c70088e4533c97deeb072363c8beef7"
Oct 30 19:17:28.227558 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:28.227541    4544 scope.go:122] "RemoveContainer" containerID="76b0fe820ae60c4adbe3d4b6e0b3f7ed4c70088e4533c97deeb072363c8beef7"
Oct 30 19:17:28.227830 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:17:28.227810    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76b0fe820ae60c4adbe3d4b6e0b3f7ed4c70088e4533c97deeb072363c8beef7\": container with ID starting with 76b0fe820ae60c4adbe3d4b6e0b3f7ed4c70088e4533c97deeb072363c8beef7 not found: ID does not exist" containerID="76b0fe820ae60c4adbe3d4b6e0b3f7ed4c70088e4533c97deeb072363c8beef7"
Oct 30 19:17:28.227923 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:28.227840    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76b0fe820ae60c4adbe3d4b6e0b3f7ed4c70088e4533c97deeb072363c8beef7"} err="failed to get container status \"76b0fe820ae60c4adbe3d4b6e0b3f7ed4c70088e4533c97deeb072363c8beef7\": rpc error: code = NotFound desc = could not find container \"76b0fe820ae60c4adbe3d4b6e0b3f7ed4c70088e4533c97deeb072363c8beef7\": container with ID starting with 76b0fe820ae60c4adbe3d4b6e0b3f7ed4c70088e4533c97deeb072363c8beef7 not found: ID does not exist"
Oct 30 19:17:28.372360 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:28.372325    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"]
Oct 30 19:17:28.418528 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:28.418502    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-ec8dab-86b7578497-cmgdq"]
Oct 30 19:17:28.602189 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:28.602156    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e45c27e0-1404-48e3-8289-6426eee88141" path="/var/lib/kubelet/pods/e45c27e0-1404-48e3-8289-6426eee88141/volumes"
Oct 30 19:17:33.891100 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:33.891066    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8"]
Oct 30 19:17:35.531644 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:17:35.531584    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:17:38.892095 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:38.892040    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8" podUID="5662c153-5928-418f-b1fa-74bd0d7a2727" containerName="payload-processing" containerID="cri-o://31cc546a806f6c86f9a5aeaf3ed4457d8f47aea0bac610f9e8cc458ccec5ff3a" gracePeriod=25
Oct 30 19:17:39.995762 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:39.995722    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8" podUID="5662c153-5928-418f-b1fa-74bd0d7a2727" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:17:41.172093 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172057    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"]
Oct 30 19:17:41.172489 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172477    4544 cpu_manager.go:414] "containerMap: removing container" podUID="fbd9d51e-133f-4b3a-b592-38ca4b2b5d06" containerName="maas-api"
Oct 30 19:17:41.172534 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172490    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbd9d51e-133f-4b3a-b592-38ca4b2b5d06" containerName="maas-api"
Oct 30 19:17:41.172534 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172505    4544 cpu_manager.go:414] "containerMap: removing container" podUID="729b4f97-7d22-49ae-ac1b-4528af3b5242" containerName="istio-proxy"
Oct 30 19:17:41.172534 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172510    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="729b4f97-7d22-49ae-ac1b-4528af3b5242" containerName="istio-proxy"
Oct 30 19:17:41.172534 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172519    4544 cpu_manager.go:414] "containerMap: removing container" podUID="6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" containerName="istio-proxy"
Oct 30 19:17:41.172534 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172524    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" containerName="istio-proxy"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172537    4544 cpu_manager.go:414] "containerMap: removing container" podUID="34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" containerName="istio-proxy"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172541    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" containerName="istio-proxy"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172549    4544 cpu_manager.go:414] "containerMap: removing container" podUID="729b4f97-7d22-49ae-ac1b-4528af3b5242" containerName="istio-proxy"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172554    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="729b4f97-7d22-49ae-ac1b-4528af3b5242" containerName="istio-proxy"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172569    4544 cpu_manager.go:414] "containerMap: removing container" podUID="15b9134c-0ca4-4285-a041-badc8cddc776" containerName="maas-api"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172574    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="15b9134c-0ca4-4285-a041-badc8cddc776" containerName="maas-api"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172586    4544 cpu_manager.go:414] "containerMap: removing container" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172591    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172612    4544 cpu_manager.go:414] "containerMap: removing container" podUID="e45c27e0-1404-48e3-8289-6426eee88141" containerName="maas-api"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172617    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e45c27e0-1404-48e3-8289-6426eee88141" containerName="maas-api"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172628    4544 cpu_manager.go:414] "containerMap: removing container" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172638    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172647    4544 cpu_manager.go:414] "containerMap: removing container" podUID="14497a85-b284-431d-930f-a0ac9274ab6c" containerName="istio-proxy"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172652    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14497a85-b284-431d-930f-a0ac9274ab6c" containerName="istio-proxy"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172657    4544 cpu_manager.go:414] "containerMap: removing container" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172661    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172667    4544 cpu_manager.go:414] "containerMap: removing container" podUID="14497a85-b284-431d-930f-a0ac9274ab6c" containerName="istio-proxy"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172671    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14497a85-b284-431d-930f-a0ac9274ab6c" containerName="istio-proxy"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172677    4544 cpu_manager.go:414] "containerMap: removing container" podUID="e4dc6f73-5287-4f71-b368-6c0d605e9054" containerName="istio-proxy"
Oct 30 19:17:41.172730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172681    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e4dc6f73-5287-4f71-b368-6c0d605e9054" containerName="istio-proxy"
Oct 30 19:17:41.173327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172740    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="15b9134c-0ca4-4285-a041-badc8cddc776" containerName="maas-api"
Oct 30 19:17:41.173327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172752    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="f71923a7-7e8a-48e0-adcd-a061c07eb212" containerName="main"
Oct 30 19:17:41.173327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172762    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="e45c27e0-1404-48e3-8289-6426eee88141" containerName="maas-api"
Oct 30 19:17:41.173327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172769    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd9777c9-1b40-42fd-9815-cdf1a3509661" containerName="main"
Oct 30 19:17:41.173327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172774    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="14497a85-b284-431d-930f-a0ac9274ab6c" containerName="istio-proxy"
Oct 30 19:17:41.173327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172778    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="bfb6e108-03a3-4d0f-94d4-2a29eee0d582" containerName="main"
Oct 30 19:17:41.173327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172784    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="14497a85-b284-431d-930f-a0ac9274ab6c" containerName="istio-proxy"
Oct 30 19:17:41.173327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172789    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="e4dc6f73-5287-4f71-b368-6c0d605e9054" containerName="istio-proxy"
Oct 30 19:17:41.173327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172795    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbd9d51e-133f-4b3a-b592-38ca4b2b5d06" containerName="maas-api"
Oct 30 19:17:41.173327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172799    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="729b4f97-7d22-49ae-ac1b-4528af3b5242" containerName="istio-proxy"
Oct 30 19:17:41.173327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172805    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="6276b3df-6da0-43fa-8bf1-2ffa76f0ce95" containerName="istio-proxy"
Oct 30 19:17:41.173327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172811    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="34f008d5-e8ec-48d2-9ffb-1d00ceb1f9fd" containerName="istio-proxy"
Oct 30 19:17:41.173327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.172816    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="729b4f97-7d22-49ae-ac1b-4528af3b5242" containerName="istio-proxy"
Oct 30 19:17:41.175971 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.175955    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.218930 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.218904    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"]
Oct 30 19:17:41.222417 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.222398    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"]
Oct 30 19:17:41.222553 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.222530    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.231769 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.231749    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.250500 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.250469    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85189f79-bfea-4791-bad5-f30a93398ea8-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.250777 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.250748    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85189f79-bfea-4791-bad5-f30a93398ea8-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.254730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.254704    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smvp7\" (UniqueName: \"kubernetes.io/projected/85189f79-bfea-4791-bad5-f30a93398ea8-kube-api-access-smvp7\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.254890 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.254875    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85189f79-bfea-4791-bad5-f30a93398ea8-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.255026 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.255004    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85189f79-bfea-4791-bad5-f30a93398ea8-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.255186 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.255080    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85189f79-bfea-4791-bad5-f30a93398ea8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.255186 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.255151    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85189f79-bfea-4791-bad5-f30a93398ea8-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.255186 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.255180    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85189f79-bfea-4791-bad5-f30a93398ea8-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.255656 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.255221    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85189f79-bfea-4791-bad5-f30a93398ea8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.263092 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.263005    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-g5j8l\""
Oct 30 19:17:41.271361 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.271334    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"]
Oct 30 19:17:41.318005 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.317976    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"]
Oct 30 19:17:41.337000 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.336970    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"]
Oct 30 19:17:41.356184 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356135    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njrv4\" (UniqueName: \"kubernetes.io/projected/28e7f033-4399-4555-8e0b-d2642bbf3837-kube-api-access-njrv4\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.356350 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356205    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28e7f033-4399-4555-8e0b-d2642bbf3837-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.356350 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356258    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28e7f033-4399-4555-8e0b-d2642bbf3837-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.356350 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356294    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85189f79-bfea-4791-bad5-f30a93398ea8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.356350 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356325    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk2lt\" (UniqueName: \"kubernetes.io/projected/e269d837-0eb0-4d79-9c11-2f411871f859-kube-api-access-bk2lt\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.356586 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356435    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85189f79-bfea-4791-bad5-f30a93398ea8-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.356586 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356460    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28e7f033-4399-4555-8e0b-d2642bbf3837-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.356586 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356476    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28e7f033-4399-4555-8e0b-d2642bbf3837-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.356586 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356535    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85189f79-bfea-4791-bad5-f30a93398ea8-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.356891 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356650    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85189f79-bfea-4791-bad5-f30a93398ea8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.356891 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356695    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e269d837-0eb0-4d79-9c11-2f411871f859-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.356891 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356727    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85189f79-bfea-4791-bad5-f30a93398ea8-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.356891 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356758    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85189f79-bfea-4791-bad5-f30a93398ea8-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.356891 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356785    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e269d837-0eb0-4d79-9c11-2f411871f859-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.356891 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356823    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e269d837-0eb0-4d79-9c11-2f411871f859-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.356891 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356857    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e269d837-0eb0-4d79-9c11-2f411871f859-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.356891 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356892    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e269d837-0eb0-4d79-9c11-2f411871f859-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.357355 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356899    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85189f79-bfea-4791-bad5-f30a93398ea8-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.357355 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.356917    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28e7f033-4399-4555-8e0b-d2642bbf3837-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.357355 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.357144    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e269d837-0eb0-4d79-9c11-2f411871f859-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.357355 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.357289    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28e7f033-4399-4555-8e0b-d2642bbf3837-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.357355 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.357299    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85189f79-bfea-4791-bad5-f30a93398ea8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.357355 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.357319    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e269d837-0eb0-4d79-9c11-2f411871f859-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.357730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.357374    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smvp7\" (UniqueName: \"kubernetes.io/projected/85189f79-bfea-4791-bad5-f30a93398ea8-kube-api-access-smvp7\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.357730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.357411    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e269d837-0eb0-4d79-9c11-2f411871f859-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.357730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.357451    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85189f79-bfea-4791-bad5-f30a93398ea8-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.357730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.357477    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28e7f033-4399-4555-8e0b-d2642bbf3837-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.357730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.357561    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28e7f033-4399-4555-8e0b-d2642bbf3837-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.357730 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.357595    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85189f79-bfea-4791-bad5-f30a93398ea8-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.358068 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.358046    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85189f79-bfea-4791-bad5-f30a93398ea8-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.358128 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.358067    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85189f79-bfea-4791-bad5-f30a93398ea8-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.358228 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.358207    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85189f79-bfea-4791-bad5-f30a93398ea8-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.359366 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.359343    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85189f79-bfea-4791-bad5-f30a93398ea8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.360376 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.360355    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85189f79-bfea-4791-bad5-f30a93398ea8-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.412037 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.412010    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smvp7\" (UniqueName: \"kubernetes.io/projected/85189f79-bfea-4791-bad5-f30a93398ea8-kube-api-access-smvp7\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.412146 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.412016    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85189f79-bfea-4791-bad5-f30a93398ea8-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-g58p2\" (UID: \"85189f79-bfea-4791-bad5-f30a93398ea8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.458558 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.458486    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28e7f033-4399-4555-8e0b-d2642bbf3837-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.458558 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.458531    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28e7f033-4399-4555-8e0b-d2642bbf3837-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.458773 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.458590    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk2lt\" (UniqueName: \"kubernetes.io/projected/e269d837-0eb0-4d79-9c11-2f411871f859-kube-api-access-bk2lt\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.458773 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.458662    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28e7f033-4399-4555-8e0b-d2642bbf3837-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.458773 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.458687    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28e7f033-4399-4555-8e0b-d2642bbf3837-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.458773 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.458739    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e269d837-0eb0-4d79-9c11-2f411871f859-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.459058 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.458778    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e269d837-0eb0-4d79-9c11-2f411871f859-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.459058 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.458814    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e269d837-0eb0-4d79-9c11-2f411871f859-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.459058 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.458847    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e269d837-0eb0-4d79-9c11-2f411871f859-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.459058 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.458877    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e269d837-0eb0-4d79-9c11-2f411871f859-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.459058 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.458903    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28e7f033-4399-4555-8e0b-d2642bbf3837-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.459058 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.458927    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e269d837-0eb0-4d79-9c11-2f411871f859-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.459058 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.458965    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28e7f033-4399-4555-8e0b-d2642bbf3837-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.459058 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.458969    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28e7f033-4399-4555-8e0b-d2642bbf3837-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.459058 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.458992    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e269d837-0eb0-4d79-9c11-2f411871f859-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.459058 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.459031    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e269d837-0eb0-4d79-9c11-2f411871f859-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.459598 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.459062    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28e7f033-4399-4555-8e0b-d2642bbf3837-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.459598 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.459115    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28e7f033-4399-4555-8e0b-d2642bbf3837-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.459598 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.459159    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njrv4\" (UniqueName: \"kubernetes.io/projected/28e7f033-4399-4555-8e0b-d2642bbf3837-kube-api-access-njrv4\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.459598 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.459210    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e269d837-0eb0-4d79-9c11-2f411871f859-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.459598 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.459261    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e269d837-0eb0-4d79-9c11-2f411871f859-workload-certs\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.459598 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.459304    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e269d837-0eb0-4d79-9c11-2f411871f859-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.459598 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.459390    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e269d837-0eb0-4d79-9c11-2f411871f859-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.459598 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.459500    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28e7f033-4399-4555-8e0b-d2642bbf3837-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.459598 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.459573    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28e7f033-4399-4555-8e0b-d2642bbf3837-istio-data\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.459991 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.459618    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28e7f033-4399-4555-8e0b-d2642bbf3837-credential-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.459991 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.459713    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28e7f033-4399-4555-8e0b-d2642bbf3837-workload-socket\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.460270 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.460249    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e269d837-0eb0-4d79-9c11-2f411871f859-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.461278 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.461255    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28e7f033-4399-4555-8e0b-d2642bbf3837-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.461833 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.461811    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e269d837-0eb0-4d79-9c11-2f411871f859-istio-envoy\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.461964 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.461946    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28e7f033-4399-4555-8e0b-d2642bbf3837-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.462026 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.462008    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e269d837-0eb0-4d79-9c11-2f411871f859-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.489000 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.488977    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:41.514975 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.514518    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk2lt\" (UniqueName: \"kubernetes.io/projected/e269d837-0eb0-4d79-9c11-2f411871f859-kube-api-access-bk2lt\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.515199 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.515176    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28e7f033-4399-4555-8e0b-d2642bbf3837-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.515427 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.515404    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njrv4\" (UniqueName: \"kubernetes.io/projected/28e7f033-4399-4555-8e0b-d2642bbf3837-kube-api-access-njrv4\") pod \"maas-default-gateway-openshift-default-75d7c785f-nf2pw\" (UID: \"28e7f033-4399-4555-8e0b-d2642bbf3837\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.515661 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.515645    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e269d837-0eb0-4d79-9c11-2f411871f859-istio-token\") pod \"maas-default-gateway-openshift-default-75d7c785f-cjv6v\" (UID: \"e269d837-0eb0-4d79-9c11-2f411871f859\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.535292 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.535269    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:41.562987 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.562960    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:41.914160 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.914125    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"]
Oct 30 19:17:41.946892 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:17:41.946864    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85189f79_bfea_4791_bad5_f30a93398ea8.slice/crio-8a09c68bffd4e3c8df28e61c6b379cb4336d61d597e7c86bb22cb8b74a062d8e WatchSource:0}: Error finding container 8a09c68bffd4e3c8df28e61c6b379cb4336d61d597e7c86bb22cb8b74a062d8e: Status 404 returned error can't find the container with id 8a09c68bffd4e3c8df28e61c6b379cb4336d61d597e7c86bb22cb8b74a062d8e
Oct 30 19:17:41.949119 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.949087    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:17:41.949216 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.949151    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:17:41.949216 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.949182    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:17:41.986334 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.986312    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"]
Oct 30 19:17:41.989663 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:17:41.989638    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode269d837_0eb0_4d79_9c11_2f411871f859.slice/crio-68f19425afa3de13c3b3d82d9e1752b956fa1b160a66346314ac952173a84fb5 WatchSource:0}: Error finding container 68f19425afa3de13c3b3d82d9e1752b956fa1b160a66346314ac952173a84fb5: Status 404 returned error can't find the container with id 68f19425afa3de13c3b3d82d9e1752b956fa1b160a66346314ac952173a84fb5
Oct 30 19:17:41.991864 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.991836    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:17:41.991948 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.991897    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:17:41.991948 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:41.991922    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:17:42.013658 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.013635    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"]
Oct 30 19:17:42.020680 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.020355    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:17:42.020680 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.020439    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:17:42.020680 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.020483    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:17:42.206834 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.206746    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main" containerID="cri-o://6e2eaa7ecc2884896fe4d5a66a2f65b01874862ec11e1bd876185344921f2f1e" gracePeriod=45
Oct 30 19:17:42.252279 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.252235    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main" containerID="cri-o://cbb186716b76112b2a444ff9e6993a8554edfc0cc22a962fbffe57033842a6dd" gracePeriod=45
Oct 30 19:17:42.305351 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.305239    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw" event={"ID":"28e7f033-4399-4555-8e0b-d2642bbf3837","Type":"ContainerStarted","Data":"d0d9cdfd16e99b9da55a34fa0ce03785bf4806981ef53fb1d964e9e3c68980e7"}
Oct 30 19:17:42.305351 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.305314    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw" event={"ID":"28e7f033-4399-4555-8e0b-d2642bbf3837","Type":"ContainerStarted","Data":"9c0b0036fabd059f73a24dcb119d54bc1470eafc420f4a8139415f6ded3eb6fc"}
Oct 30 19:17:42.307709 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.307678    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2" event={"ID":"85189f79-bfea-4791-bad5-f30a93398ea8","Type":"ContainerStarted","Data":"fd22c1befbf7ca9dfc53fbc8a235398e1004a7a8fae4ecdb102eac6c8cd1cd69"}
Oct 30 19:17:42.307848 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.307729    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2" event={"ID":"85189f79-bfea-4791-bad5-f30a93398ea8","Type":"ContainerStarted","Data":"8a09c68bffd4e3c8df28e61c6b379cb4336d61d597e7c86bb22cb8b74a062d8e"}
Oct 30 19:17:42.310088 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.310064    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v" event={"ID":"e269d837-0eb0-4d79-9c11-2f411871f859","Type":"ContainerStarted","Data":"af3e9a9464adb04a90e5faf6799660d53e342d5fb51dbcf62ea69285d733d561"}
Oct 30 19:17:42.310184 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.310109    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v" event={"ID":"e269d837-0eb0-4d79-9c11-2f411871f859","Type":"ContainerStarted","Data":"68f19425afa3de13c3b3d82d9e1752b956fa1b160a66346314ac952173a84fb5"}
Oct 30 19:17:42.430891 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.428531    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw" podStartSLOduration=1.428512277 podStartE2EDuration="1.428512277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:17:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:17:42.426734592 +0000 UTC m=+2459.952609688" watchObservedRunningTime="2026-10-30 19:17:42.428512277 +0000 UTC m=+2459.954387374"
Oct 30 19:17:42.491410 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.489879    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:42.540723 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.537735    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:42.565649 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.562828    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v" podStartSLOduration=1.562806831 podStartE2EDuration="1.562806831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:17:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:17:42.557975467 +0000 UTC m=+2460.083850564" watchObservedRunningTime="2026-10-30 19:17:42.562806831 +0000 UTC m=+2460.088681928"
Oct 30 19:17:42.565649 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.563446    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:42.734798 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.734580    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 19:17:42.762852 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.762820    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 19:17:42.837979 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.837942    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79574f74-b0c4-4584-8f18-84f1659e020b-kube-api-access-zmsjm\" (UniqueName: \"kubernetes.io/projected/79574f74-b0c4-4584-8f18-84f1659e020b-kube-api-access-zmsjm\") pod \"79574f74-b0c4-4584-8f18-84f1659e020b\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") "
Oct 30 19:17:42.838318 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.838298    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-model-cache\") pod \"79574f74-b0c4-4584-8f18-84f1659e020b\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") "
Oct 30 19:17:42.844629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.838451    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-home\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-home\") pod \"79574f74-b0c4-4584-8f18-84f1659e020b\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") "
Oct 30 19:17:42.865629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.848694    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-tmp-dir\") pod \"79574f74-b0c4-4584-8f18-84f1659e020b\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") "
Oct 30 19:17:42.865629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.855795    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-dshm\") pod \"79574f74-b0c4-4584-8f18-84f1659e020b\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") "
Oct 30 19:17:42.865629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.855891    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/79574f74-b0c4-4584-8f18-84f1659e020b-tls-certs\" (UniqueName: \"kubernetes.io/secret/79574f74-b0c4-4584-8f18-84f1659e020b-tls-certs\") pod \"79574f74-b0c4-4584-8f18-84f1659e020b\" (UID: \"79574f74-b0c4-4584-8f18-84f1659e020b\") "
Oct 30 19:17:42.909660 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.897321    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2" podStartSLOduration=1.89729831 podStartE2EDuration="1.89729831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:17:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:17:42.709030938 +0000 UTC m=+2460.234906035" watchObservedRunningTime="2026-10-30 19:17:42.89729831 +0000 UTC m=+2460.423173406"
Oct 30 19:17:42.909660 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.905699    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79574f74-b0c4-4584-8f18-84f1659e020b-tls-certs" pod "79574f74-b0c4-4584-8f18-84f1659e020b" (UID: "79574f74-b0c4-4584-8f18-84f1659e020b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:17:42.909660 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.906379    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-home" pod "79574f74-b0c4-4584-8f18-84f1659e020b" (UID: "79574f74-b0c4-4584-8f18-84f1659e020b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:42.909660 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.906557    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-model-cache" pod "79574f74-b0c4-4584-8f18-84f1659e020b" (UID: "79574f74-b0c4-4584-8f18-84f1659e020b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:42.909660 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.906861    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-tmp-dir" pod "79574f74-b0c4-4584-8f18-84f1659e020b" (UID: "79574f74-b0c4-4584-8f18-84f1659e020b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:42.917990 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.917952    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79574f74-b0c4-4584-8f18-84f1659e020b-kube-api-access-zmsjm" pod "79574f74-b0c4-4584-8f18-84f1659e020b" (UID: "79574f74-b0c4-4584-8f18-84f1659e020b"). InnerVolumeSpecName "kube-api-access-zmsjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:42.961626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.956830    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-model-cache\") pod \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") "
Oct 30 19:17:42.966765 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.966729    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-kube-api-access-r7rsv\" (UniqueName: \"kubernetes.io/projected/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-kube-api-access-r7rsv\") pod \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") "
Oct 30 19:17:42.967136 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.967110    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-tls-certs\" (UniqueName: \"kubernetes.io/secret/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-tls-certs\") pod \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") "
Oct 30 19:17:42.970755 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.970729    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-dshm\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-dshm\") pod \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") "
Oct 30 19:17:42.971025 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.971008    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-home\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-home\") pod \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") "
Oct 30 19:17:42.974625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.971155    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-tmp-dir\") pod \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\" (UID: \"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06\") "
Oct 30 19:17:42.975343 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.975319    4544 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79574f74-b0c4-4584-8f18-84f1659e020b-tls-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:42.975478 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.975464    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmsjm\" (UniqueName: \"kubernetes.io/projected/79574f74-b0c4-4584-8f18-84f1659e020b-kube-api-access-zmsjm\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:42.975623 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.975588    4544 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-model-cache\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:42.975738 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.975724    4544 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-home\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:42.975822 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.975811    4544 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-tmp-dir\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:42.975894 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.967302    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-model-cache" pod "17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" (UID: "17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:42.975980 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.972740    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-dshm" pod "79574f74-b0c4-4584-8f18-84f1659e020b" (UID: "79574f74-b0c4-4584-8f18-84f1659e020b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:42.986738 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.976710    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-tmp-dir" pod "17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" (UID: "17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:42.987195 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:42.987167    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-home" pod "17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" (UID: "17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:43.054432 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.054016    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-tls-certs" pod "17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" (UID: "17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:17:43.074642 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.055742    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-dshm" pod "17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" (UID: "17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:43.074642 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.055911    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-kube-api-access-r7rsv" pod "17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" (UID: "17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06"). InnerVolumeSpecName "kube-api-access-r7rsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:43.094681 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.086729    4544 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-tls-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:43.094681 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.086764    4544 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-dshm\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:43.094681 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.086778    4544 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-home\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:43.094681 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.086791    4544 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79574f74-b0c4-4584-8f18-84f1659e020b-dshm\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:43.094681 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.086803    4544 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-tmp-dir\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:43.094681 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.086817    4544 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-model-cache\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:43.094681 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.086832    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7rsv\" (UniqueName: \"kubernetes.io/projected/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06-kube-api-access-r7rsv\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:43.343974 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.343928    4544 generic.go:356] "Generic (PLEG): container finished" podID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerID="6e2eaa7ecc2884896fe4d5a66a2f65b01874862ec11e1bd876185344921f2f1e" exitCode=0
Oct 30 19:17:43.344440 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.343998    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"
Oct 30 19:17:43.344440 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.344078    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" event={"ID":"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06","Type":"ContainerDied","Data":"6e2eaa7ecc2884896fe4d5a66a2f65b01874862ec11e1bd876185344921f2f1e"}
Oct 30 19:17:43.344440 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.344122    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s" event={"ID":"17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06","Type":"ContainerDied","Data":"894642a5e73df072b906a1ec08ef88bb887dd831bd0240dae61ba0c5434e6c01"}
Oct 30 19:17:43.344440 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.344141    4544 scope.go:122] "RemoveContainer" containerID="6e2eaa7ecc2884896fe4d5a66a2f65b01874862ec11e1bd876185344921f2f1e"
Oct 30 19:17:43.345896 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.345872    4544 generic.go:356] "Generic (PLEG): container finished" podID="79574f74-b0c4-4584-8f18-84f1659e020b" containerID="cbb186716b76112b2a444ff9e6993a8554edfc0cc22a962fbffe57033842a6dd" exitCode=0
Oct 30 19:17:43.346194 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.346178    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"
Oct 30 19:17:43.346335 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.346308    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" event={"ID":"79574f74-b0c4-4584-8f18-84f1659e020b","Type":"ContainerDied","Data":"cbb186716b76112b2a444ff9e6993a8554edfc0cc22a962fbffe57033842a6dd"}
Oct 30 19:17:43.346459 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.346442    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n" event={"ID":"79574f74-b0c4-4584-8f18-84f1659e020b","Type":"ContainerDied","Data":"ec476950021d533184d3889c534cffd5f6347c2d77b5fcb4660588bfb44d7a43"}
Oct 30 19:17:43.354474 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.354456    4544 scope.go:122] "RemoveContainer" containerID="6e2eaa7ecc2884896fe4d5a66a2f65b01874862ec11e1bd876185344921f2f1e"
Oct 30 19:17:43.354878 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:17:43.354857    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e2eaa7ecc2884896fe4d5a66a2f65b01874862ec11e1bd876185344921f2f1e\": container with ID starting with 6e2eaa7ecc2884896fe4d5a66a2f65b01874862ec11e1bd876185344921f2f1e not found: ID does not exist" containerID="6e2eaa7ecc2884896fe4d5a66a2f65b01874862ec11e1bd876185344921f2f1e"
Oct 30 19:17:43.354992 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.354951    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e2eaa7ecc2884896fe4d5a66a2f65b01874862ec11e1bd876185344921f2f1e"} err="failed to get container status \"6e2eaa7ecc2884896fe4d5a66a2f65b01874862ec11e1bd876185344921f2f1e\": rpc error: code = NotFound desc = could not find container \"6e2eaa7ecc2884896fe4d5a66a2f65b01874862ec11e1bd876185344921f2f1e\": container with ID starting with 6e2eaa7ecc2884896fe4d5a66a2f65b01874862ec11e1bd876185344921f2f1e not found: ID does not exist"
Oct 30 19:17:43.354992 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.354970    4544 scope.go:122] "RemoveContainer" containerID="cbb186716b76112b2a444ff9e6993a8554edfc0cc22a962fbffe57033842a6dd"
Oct 30 19:17:43.447281 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.447258    4544 scope.go:122] "RemoveContainer" containerID="cbb186716b76112b2a444ff9e6993a8554edfc0cc22a962fbffe57033842a6dd"
Oct 30 19:17:43.447700 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:17:43.447673    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbb186716b76112b2a444ff9e6993a8554edfc0cc22a962fbffe57033842a6dd\": container with ID starting with cbb186716b76112b2a444ff9e6993a8554edfc0cc22a962fbffe57033842a6dd not found: ID does not exist" containerID="cbb186716b76112b2a444ff9e6993a8554edfc0cc22a962fbffe57033842a6dd"
Oct 30 19:17:43.447807 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.447710    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbb186716b76112b2a444ff9e6993a8554edfc0cc22a962fbffe57033842a6dd"} err="failed to get container status \"cbb186716b76112b2a444ff9e6993a8554edfc0cc22a962fbffe57033842a6dd\": rpc error: code = NotFound desc = could not find container \"cbb186716b76112b2a444ff9e6993a8554edfc0cc22a962fbffe57033842a6dd\": container with ID starting with cbb186716b76112b2a444ff9e6993a8554edfc0cc22a962fbffe57033842a6dd not found: ID does not exist"
Oct 30 19:17:43.513987 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.497081    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-g58p2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 19:17:43.513987 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.497169    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2" podUID="85189f79-bfea-4791-bad5-f30a93398ea8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 19:17:43.538904 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.538877    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-cjv6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 30 19:17:43.539051 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.539033    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v" podUID="e269d837-0eb0-4d79-9c11-2f411871f859" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 30 19:17:43.604415 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.604299    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-nf2pw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 19:17:43.604734 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.604686    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw" podUID="28e7f033-4399-4555-8e0b-d2642bbf3837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 19:17:43.633509 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.633473    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"]
Oct 30 19:17:43.638319 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.638286    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-3b6ad6/distinct-2-w5-3b6ad6-kserve-679c5b4665-28k6s"]
Oct 30 19:17:43.714629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.714368    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"]
Oct 30 19:17:43.737628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:43.737342    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-3b6ad6/premium-w5-3b6ad6-kserve-5bc685bb55-mlz7n"]
Oct 30 19:17:44.493083 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:44.493048    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-g58p2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 19:17:44.493691 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:44.493655    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2" podUID="85189f79-bfea-4791-bad5-f30a93398ea8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 19:17:44.538044 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:44.538007    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-cjv6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 30 19:17:44.538505 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:44.538476    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v" podUID="e269d837-0eb0-4d79-9c11-2f411871f859" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 30 19:17:44.567628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:44.564975    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-nf2pw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 19:17:44.567628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:44.565031    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw" podUID="28e7f033-4399-4555-8e0b-d2642bbf3837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 19:17:44.641100 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:44.641066    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" path="/var/lib/kubelet/pods/17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06/volumes"
Oct 30 19:17:44.641902 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:44.641882    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" path="/var/lib/kubelet/pods/79574f74-b0c4-4584-8f18-84f1659e020b/volumes"
Oct 30 19:17:45.490081 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:45.490042    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-g58p2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 19:17:45.490378 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:45.490326    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2" podUID="85189f79-bfea-4791-bad5-f30a93398ea8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 19:17:45.538994 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:45.538956    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-cjv6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 30 19:17:45.539555 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:45.539513    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v" podUID="e269d837-0eb0-4d79-9c11-2f411871f859" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 30 19:17:45.567625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:45.565018    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-nf2pw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 19:17:45.567625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:45.565077    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw" podUID="28e7f033-4399-4555-8e0b-d2642bbf3837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 19:17:46.492053 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:46.492016    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-g58p2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 19:17:46.492353 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:46.492323    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2" podUID="85189f79-bfea-4791-bad5-f30a93398ea8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 19:17:46.540628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:46.535987    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-cjv6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 30 19:17:46.540628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:46.536051    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v" podUID="e269d837-0eb0-4d79-9c11-2f411871f859" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 30 19:17:46.566628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:46.566165    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-nf2pw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 19:17:46.566628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:46.566224    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw" podUID="28e7f033-4399-4555-8e0b-d2642bbf3837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 19:17:47.490626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:47.490007    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-g58p2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 19:17:47.490626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:47.490077    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2" podUID="85189f79-bfea-4791-bad5-f30a93398ea8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 19:17:47.536628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:47.536006    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-cjv6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 30 19:17:47.536628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:47.536072    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v" podUID="e269d837-0eb0-4d79-9c11-2f411871f859" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 30 19:17:47.573629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:47.563964    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-nf2pw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 19:17:47.573629 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:47.564023    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw" podUID="28e7f033-4399-4555-8e0b-d2642bbf3837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 19:17:48.489837 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:48.489799    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-g58p2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 19:17:48.490013 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:48.489867    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2" podUID="85189f79-bfea-4791-bad5-f30a93398ea8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 19:17:48.536492 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:48.536451    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-cjv6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 30 19:17:48.536701 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:48.536498    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v" podUID="e269d837-0eb0-4d79-9c11-2f411871f859" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 30 19:17:48.563782 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:48.563751    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-nf2pw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 19:17:48.564016 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:48.563804    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw" podUID="28e7f033-4399-4555-8e0b-d2642bbf3837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 19:17:49.489534 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:49.489493    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-g58p2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused" start-of-body=
Oct 30 19:17:49.490038 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:49.489558    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2" podUID="85189f79-bfea-4791-bad5-f30a93398ea8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.124:15021/healthz/ready\": dial tcp 10.133.0.124:15021: connect: connection refused"
Oct 30 19:17:49.536373 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:49.536335    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-cjv6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 30 19:17:49.536581 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:49.536400    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v" podUID="e269d837-0eb0-4d79-9c11-2f411871f859" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 30 19:17:49.564200 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:49.564166    4544 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75d7c785f-nf2pw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 30 19:17:49.564341 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:49.564227    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw" podUID="28e7f033-4399-4555-8e0b-d2642bbf3837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 30 19:17:49.995753 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:49.995712    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8" podUID="5662c153-5928-418f-b1fa-74bd0d7a2727" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:17:51.312694 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:51.312471    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:51.313546 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:51.313330    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:51.316113 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:51.316080    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-g58p2"
Oct 30 19:17:51.324398 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:51.324376    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:51.325142 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:51.325120    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:51.326014 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:51.325995    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-cjv6v"
Oct 30 19:17:51.374767 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:51.374739    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:51.375042 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:51.375017    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:51.375812 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:51.375792    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75d7c785f-nf2pw"
Oct 30 19:17:59.152778 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.152713    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8"
Oct 30 19:17:59.209493 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.209456    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5662c153-5928-418f-b1fa-74bd0d7a2727-kube-api-access-2lkgd\" (UniqueName: \"kubernetes.io/projected/5662c153-5928-418f-b1fa-74bd0d7a2727-kube-api-access-2lkgd\") pod \"5662c153-5928-418f-b1fa-74bd0d7a2727\" (UID: \"5662c153-5928-418f-b1fa-74bd0d7a2727\") "
Oct 30 19:17:59.209721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.209542    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5662c153-5928-418f-b1fa-74bd0d7a2727-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5662c153-5928-418f-b1fa-74bd0d7a2727-plugins-config-volume\") pod \"5662c153-5928-418f-b1fa-74bd0d7a2727\" (UID: \"5662c153-5928-418f-b1fa-74bd0d7a2727\") "
Oct 30 19:17:59.209985 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.209958    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5662c153-5928-418f-b1fa-74bd0d7a2727-plugins-config-volume" pod "5662c153-5928-418f-b1fa-74bd0d7a2727" (UID: "5662c153-5928-418f-b1fa-74bd0d7a2727"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:59.211739 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.211718    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5662c153-5928-418f-b1fa-74bd0d7a2727-kube-api-access-2lkgd" pod "5662c153-5928-418f-b1fa-74bd0d7a2727" (UID: "5662c153-5928-418f-b1fa-74bd0d7a2727"). InnerVolumeSpecName "kube-api-access-2lkgd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:59.311137 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.311099    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5662c153-5928-418f-b1fa-74bd0d7a2727-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:59.311137 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.311133    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lkgd\" (UniqueName: \"kubernetes.io/projected/5662c153-5928-418f-b1fa-74bd0d7a2727-kube-api-access-2lkgd\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:59.425538 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.425451    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"]
Oct 30 19:17:59.425809 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.425760    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm" podUID="dddf9498-52dc-4712-933a-fee14fc5eca9" containerName="maas-api" containerID="cri-o://341e8e7ca2158530061a3b9127f32999d6ebe40c85f8c4b61c352b14b08cd07c" gracePeriod=30
Oct 30 19:17:59.543106 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.543075    4544 generic.go:356] "Generic (PLEG): container finished" podID="dddf9498-52dc-4712-933a-fee14fc5eca9" containerID="341e8e7ca2158530061a3b9127f32999d6ebe40c85f8c4b61c352b14b08cd07c" exitCode=0
Oct 30 19:17:59.543281 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.543147    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm" event={"ID":"dddf9498-52dc-4712-933a-fee14fc5eca9","Type":"ContainerDied","Data":"341e8e7ca2158530061a3b9127f32999d6ebe40c85f8c4b61c352b14b08cd07c"}
Oct 30 19:17:59.544364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.544337    4544 generic.go:356] "Generic (PLEG): container finished" podID="5662c153-5928-418f-b1fa-74bd0d7a2727" containerID="31cc546a806f6c86f9a5aeaf3ed4457d8f47aea0bac610f9e8cc458ccec5ff3a" exitCode=0
Oct 30 19:17:59.544483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.544373    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8" event={"ID":"5662c153-5928-418f-b1fa-74bd0d7a2727","Type":"ContainerDied","Data":"31cc546a806f6c86f9a5aeaf3ed4457d8f47aea0bac610f9e8cc458ccec5ff3a"}
Oct 30 19:17:59.544483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.544393    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8"
Oct 30 19:17:59.544483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.544422    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8" event={"ID":"5662c153-5928-418f-b1fa-74bd0d7a2727","Type":"ContainerDied","Data":"86caf8befac7394734dae9f1bd7ca262864717457069a4942dca97ebfe6342ce"}
Oct 30 19:17:59.544483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.544449    4544 scope.go:122] "RemoveContainer" containerID="31cc546a806f6c86f9a5aeaf3ed4457d8f47aea0bac610f9e8cc458ccec5ff3a"
Oct 30 19:17:59.553086 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.553067    4544 scope.go:122] "RemoveContainer" containerID="31cc546a806f6c86f9a5aeaf3ed4457d8f47aea0bac610f9e8cc458ccec5ff3a"
Oct 30 19:17:59.553361 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:17:59.553341    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31cc546a806f6c86f9a5aeaf3ed4457d8f47aea0bac610f9e8cc458ccec5ff3a\": container with ID starting with 31cc546a806f6c86f9a5aeaf3ed4457d8f47aea0bac610f9e8cc458ccec5ff3a not found: ID does not exist" containerID="31cc546a806f6c86f9a5aeaf3ed4457d8f47aea0bac610f9e8cc458ccec5ff3a"
Oct 30 19:17:59.553424 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.553369    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31cc546a806f6c86f9a5aeaf3ed4457d8f47aea0bac610f9e8cc458ccec5ff3a"} err="failed to get container status \"31cc546a806f6c86f9a5aeaf3ed4457d8f47aea0bac610f9e8cc458ccec5ff3a\": rpc error: code = NotFound desc = could not find container \"31cc546a806f6c86f9a5aeaf3ed4457d8f47aea0bac610f9e8cc458ccec5ff3a\": container with ID starting with 31cc546a806f6c86f9a5aeaf3ed4457d8f47aea0bac610f9e8cc458ccec5ff3a not found: ID does not exist"
Oct 30 19:17:59.691782 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.691755    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 19:17:59.701983 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.701960    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8"]
Oct 30 19:17:59.711068 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.711045    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-3b6ad6-58855dc9d8-c4mk8"]
Oct 30 19:17:59.714922 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.714902    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dddf9498-52dc-4712-933a-fee14fc5eca9-kube-api-access-crzww\" (UniqueName: \"kubernetes.io/projected/dddf9498-52dc-4712-933a-fee14fc5eca9-kube-api-access-crzww\") pod \"dddf9498-52dc-4712-933a-fee14fc5eca9\" (UID: \"dddf9498-52dc-4712-933a-fee14fc5eca9\") "
Oct 30 19:17:59.714997 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.714947    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dddf9498-52dc-4712-933a-fee14fc5eca9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/dddf9498-52dc-4712-933a-fee14fc5eca9-metrics-certs\") pod \"dddf9498-52dc-4712-933a-fee14fc5eca9\" (UID: \"dddf9498-52dc-4712-933a-fee14fc5eca9\") "
Oct 30 19:17:59.714997 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.714986    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dddf9498-52dc-4712-933a-fee14fc5eca9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dddf9498-52dc-4712-933a-fee14fc5eca9-maas-api-tls\") pod \"dddf9498-52dc-4712-933a-fee14fc5eca9\" (UID: \"dddf9498-52dc-4712-933a-fee14fc5eca9\") "
Oct 30 19:17:59.715125 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.715110    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dddf9498-52dc-4712-933a-fee14fc5eca9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dddf9498-52dc-4712-933a-fee14fc5eca9-openshift-service-ca\") pod \"dddf9498-52dc-4712-933a-fee14fc5eca9\" (UID: \"dddf9498-52dc-4712-933a-fee14fc5eca9\") "
Oct 30 19:17:59.715681 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.715640    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dddf9498-52dc-4712-933a-fee14fc5eca9-openshift-service-ca" pod "dddf9498-52dc-4712-933a-fee14fc5eca9" (UID: "dddf9498-52dc-4712-933a-fee14fc5eca9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:59.715816 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.715794    4544 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dddf9498-52dc-4712-933a-fee14fc5eca9-openshift-service-ca\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:59.717593 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.717550    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dddf9498-52dc-4712-933a-fee14fc5eca9-maas-api-tls" pod "dddf9498-52dc-4712-933a-fee14fc5eca9" (UID: "dddf9498-52dc-4712-933a-fee14fc5eca9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:17:59.717711 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.717687    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dddf9498-52dc-4712-933a-fee14fc5eca9-kube-api-access-crzww" pod "dddf9498-52dc-4712-933a-fee14fc5eca9" (UID: "dddf9498-52dc-4712-933a-fee14fc5eca9"). InnerVolumeSpecName "kube-api-access-crzww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:59.717773 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.717747    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dddf9498-52dc-4712-933a-fee14fc5eca9-metrics-certs" pod "dddf9498-52dc-4712-933a-fee14fc5eca9" (UID: "dddf9498-52dc-4712-933a-fee14fc5eca9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:17:59.820770 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.820727    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crzww\" (UniqueName: \"kubernetes.io/projected/dddf9498-52dc-4712-933a-fee14fc5eca9-kube-api-access-crzww\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:59.820770 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.820771    4544 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dddf9498-52dc-4712-933a-fee14fc5eca9-metrics-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:59.821004 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:17:59.820787    4544 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dddf9498-52dc-4712-933a-fee14fc5eca9-maas-api-tls\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:00.551899 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:00.551855    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm" event={"ID":"dddf9498-52dc-4712-933a-fee14fc5eca9","Type":"ContainerDied","Data":"b33a001daa29135722a536775653434752518a3e921f9f0ec53fe20fcc5910d9"}
Oct 30 19:18:00.552376 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:00.551908    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"
Oct 30 19:18:00.552376 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:00.551933    4544 scope.go:122] "RemoveContainer" containerID="341e8e7ca2158530061a3b9127f32999d6ebe40c85f8c4b61c352b14b08cd07c"
Oct 30 19:18:00.601287 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:00.601259    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5662c153-5928-418f-b1fa-74bd0d7a2727" path="/var/lib/kubelet/pods/5662c153-5928-418f-b1fa-74bd0d7a2727/volumes"
Oct 30 19:18:00.707826 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:00.707791    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"]
Oct 30 19:18:00.755719 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:00.755678    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-3b6ad6-7cc4f44467-ws6rm"]
Oct 30 19:18:02.601292 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:02.601255    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dddf9498-52dc-4712-933a-fee14fc5eca9" path="/var/lib/kubelet/pods/dddf9498-52dc-4712-933a-fee14fc5eca9/volumes"
Oct 30 19:18:05.532386 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:18:05.532344    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:18:08.970359 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:08.970319    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"]
Oct 30 19:18:08.970909 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:08.970568    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5" podUID="9dd18fca-58d5-45d5-8894-1c12386726fa" containerName="istio-proxy" containerID="cri-o://6b8cd4f8cb980e52b94c6fd6cad639efbf3f3c2c7a1c553c9ee9a8c9f34d2e33" gracePeriod=30
Oct 30 19:18:14.224108 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.224081    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 19:18:14.381351 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.381064    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-envoy\") pod \"9dd18fca-58d5-45d5-8894-1c12386726fa\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") "
Oct 30 19:18:14.381351 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.381113    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-workload-socket\") pod \"9dd18fca-58d5-45d5-8894-1c12386726fa\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") "
Oct 30 19:18:14.381991 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.381959    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-data\") pod \"9dd18fca-58d5-45d5-8894-1c12386726fa\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") "
Oct 30 19:18:14.382108 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.382057    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-podinfo\") pod \"9dd18fca-58d5-45d5-8894-1c12386726fa\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") "
Oct 30 19:18:14.382108 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.382088    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-credential-socket\") pod \"9dd18fca-58d5-45d5-8894-1c12386726fa\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") "
Oct 30 19:18:14.382195 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.382154    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9dd18fca-58d5-45d5-8894-1c12386726fa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dd18fca-58d5-45d5-8894-1c12386726fa-istiod-ca-cert\") pod \"9dd18fca-58d5-45d5-8894-1c12386726fa\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") "
Oct 30 19:18:14.382249 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.382229    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-token\" (UniqueName: \"kubernetes.io/projected/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-token\") pod \"9dd18fca-58d5-45d5-8894-1c12386726fa\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") "
Oct 30 19:18:14.382307 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.382260    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9dd18fca-58d5-45d5-8894-1c12386726fa-kube-api-access-9l4t7\" (UniqueName: \"kubernetes.io/projected/9dd18fca-58d5-45d5-8894-1c12386726fa-kube-api-access-9l4t7\") pod \"9dd18fca-58d5-45d5-8894-1c12386726fa\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") "
Oct 30 19:18:14.382360 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.382308    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-workload-certs\") pod \"9dd18fca-58d5-45d5-8894-1c12386726fa\" (UID: \"9dd18fca-58d5-45d5-8894-1c12386726fa\") "
Oct 30 19:18:14.383031 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.383004    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-workload-certs" pod "9dd18fca-58d5-45d5-8894-1c12386726fa" (UID: "9dd18fca-58d5-45d5-8894-1c12386726fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:14.384242 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.383927    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-credential-socket" pod "9dd18fca-58d5-45d5-8894-1c12386726fa" (UID: "9dd18fca-58d5-45d5-8894-1c12386726fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:14.384242 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.383994    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-data" pod "9dd18fca-58d5-45d5-8894-1c12386726fa" (UID: "9dd18fca-58d5-45d5-8894-1c12386726fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:14.384242 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.384207    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-workload-socket" pod "9dd18fca-58d5-45d5-8894-1c12386726fa" (UID: "9dd18fca-58d5-45d5-8894-1c12386726fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:14.384816 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.384795    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dd18fca-58d5-45d5-8894-1c12386726fa-istiod-ca-cert" pod "9dd18fca-58d5-45d5-8894-1c12386726fa" (UID: "9dd18fca-58d5-45d5-8894-1c12386726fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:14.387514 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.387483    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-podinfo" pod "9dd18fca-58d5-45d5-8894-1c12386726fa" (UID: "9dd18fca-58d5-45d5-8894-1c12386726fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:18:14.387514 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.387511    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dd18fca-58d5-45d5-8894-1c12386726fa-kube-api-access-9l4t7" pod "9dd18fca-58d5-45d5-8894-1c12386726fa" (UID: "9dd18fca-58d5-45d5-8894-1c12386726fa"). InnerVolumeSpecName "kube-api-access-9l4t7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:14.387733 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.387492    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-envoy" pod "9dd18fca-58d5-45d5-8894-1c12386726fa" (UID: "9dd18fca-58d5-45d5-8894-1c12386726fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:14.388474 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.388446    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-token" pod "9dd18fca-58d5-45d5-8894-1c12386726fa" (UID: "9dd18fca-58d5-45d5-8894-1c12386726fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:14.483714 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.483671    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:14.483714 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.483700    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:14.483714 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.483711    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:14.483714 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.483722    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:14.484079 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.483730    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:14.484079 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.483740    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dd18fca-58d5-45d5-8894-1c12386726fa-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:14.484079 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.483749    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dd18fca-58d5-45d5-8894-1c12386726fa-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:14.484079 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.483757    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dd18fca-58d5-45d5-8894-1c12386726fa-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:14.484079 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.483766    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9l4t7\" (UniqueName: \"kubernetes.io/projected/9dd18fca-58d5-45d5-8894-1c12386726fa-kube-api-access-9l4t7\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:14.637411 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.637327    4544 generic.go:356] "Generic (PLEG): container finished" podID="9dd18fca-58d5-45d5-8894-1c12386726fa" containerID="6b8cd4f8cb980e52b94c6fd6cad639efbf3f3c2c7a1c553c9ee9a8c9f34d2e33" exitCode=0
Oct 30 19:18:14.637411 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.637374    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"
Oct 30 19:18:14.637615 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.637409    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5" event={"ID":"9dd18fca-58d5-45d5-8894-1c12386726fa","Type":"ContainerDied","Data":"6b8cd4f8cb980e52b94c6fd6cad639efbf3f3c2c7a1c553c9ee9a8c9f34d2e33"}
Oct 30 19:18:14.637615 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.637464    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5" event={"ID":"9dd18fca-58d5-45d5-8894-1c12386726fa","Type":"ContainerDied","Data":"f94252e68939df7c08583db4ece433cfd689cb1b58f267f93cc376af9381f307"}
Oct 30 19:18:14.637615 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.637488    4544 scope.go:122] "RemoveContainer" containerID="6b8cd4f8cb980e52b94c6fd6cad639efbf3f3c2c7a1c553c9ee9a8c9f34d2e33"
Oct 30 19:18:14.646185 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.646161    4544 scope.go:122] "RemoveContainer" containerID="6b8cd4f8cb980e52b94c6fd6cad639efbf3f3c2c7a1c553c9ee9a8c9f34d2e33"
Oct 30 19:18:14.646456 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:18:14.646436    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b8cd4f8cb980e52b94c6fd6cad639efbf3f3c2c7a1c553c9ee9a8c9f34d2e33\": container with ID starting with 6b8cd4f8cb980e52b94c6fd6cad639efbf3f3c2c7a1c553c9ee9a8c9f34d2e33 not found: ID does not exist" containerID="6b8cd4f8cb980e52b94c6fd6cad639efbf3f3c2c7a1c553c9ee9a8c9f34d2e33"
Oct 30 19:18:14.646505 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.646466    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b8cd4f8cb980e52b94c6fd6cad639efbf3f3c2c7a1c553c9ee9a8c9f34d2e33"} err="failed to get container status \"6b8cd4f8cb980e52b94c6fd6cad639efbf3f3c2c7a1c553c9ee9a8c9f34d2e33\": rpc error: code = NotFound desc = could not find container \"6b8cd4f8cb980e52b94c6fd6cad639efbf3f3c2c7a1c553c9ee9a8c9f34d2e33\": container with ID starting with 6b8cd4f8cb980e52b94c6fd6cad639efbf3f3c2c7a1c553c9ee9a8c9f34d2e33 not found: ID does not exist"
Oct 30 19:18:14.794194 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.794159    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"]
Oct 30 19:18:14.839299 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:14.839259    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-3b6ad6-openshift-default-654b7b54c9-vbbg5"]
Oct 30 19:18:16.602375 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:18:16.602337    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9dd18fca-58d5-45d5-8894-1c12386726fa" path="/var/lib/kubelet/pods/9dd18fca-58d5-45d5-8894-1c12386726fa/volumes"
Oct 30 19:18:35.533516 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:18:35.533470    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:19:05.530353 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:19:05.530315    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:19:35.530807 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:19:35.530767    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:20:02.103467 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:02.103422    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq"]
Oct 30 19:20:05.534525 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:20:05.534488    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:20:07.103952 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:07.103890    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq" podUID="8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3" containerName="payload-pre-processing" containerID="cri-o://2beb9dd333097b62ffa5094802ee984b37f78e57b87df22d8fdaf841f8481d89" gracePeriod=25
Oct 30 19:20:13.860690 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:13.860643    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq" podUID="8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:20:17.438291 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:17.438260    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"]
Oct 30 19:20:17.438709 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:17.438433    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k" podUID="22626496-4847-4999-a9b4-97b374162f1c" containerName="maas-api" containerID="cri-o://7b9cc64ef8e1bafbcba74ae631df78b1d049d08fd3f4e637825ea5f85074772e" gracePeriod=30
Oct 30 19:20:18.695133 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:18.695106    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"
Oct 30 19:20:18.815988 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:18.815954    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/22626496-4847-4999-a9b4-97b374162f1c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22626496-4847-4999-a9b4-97b374162f1c-maas-api-tls\") pod \"22626496-4847-4999-a9b4-97b374162f1c\" (UID: \"22626496-4847-4999-a9b4-97b374162f1c\") "
Oct 30 19:20:18.816146 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:18.816079    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/22626496-4847-4999-a9b4-97b374162f1c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/22626496-4847-4999-a9b4-97b374162f1c-metrics-certs\") pod \"22626496-4847-4999-a9b4-97b374162f1c\" (UID: \"22626496-4847-4999-a9b4-97b374162f1c\") "
Oct 30 19:20:18.816225 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:18.816185    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22626496-4847-4999-a9b4-97b374162f1c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22626496-4847-4999-a9b4-97b374162f1c-openshift-service-ca\") pod \"22626496-4847-4999-a9b4-97b374162f1c\" (UID: \"22626496-4847-4999-a9b4-97b374162f1c\") "
Oct 30 19:20:18.816287 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:18.816230    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22626496-4847-4999-a9b4-97b374162f1c-kube-api-access-k2phd\" (UniqueName: \"kubernetes.io/projected/22626496-4847-4999-a9b4-97b374162f1c-kube-api-access-k2phd\") pod \"22626496-4847-4999-a9b4-97b374162f1c\" (UID: \"22626496-4847-4999-a9b4-97b374162f1c\") "
Oct 30 19:20:18.816598 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:18.816568    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22626496-4847-4999-a9b4-97b374162f1c-openshift-service-ca" pod "22626496-4847-4999-a9b4-97b374162f1c" (UID: "22626496-4847-4999-a9b4-97b374162f1c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:18.818179 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:18.818153    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22626496-4847-4999-a9b4-97b374162f1c-maas-api-tls" pod "22626496-4847-4999-a9b4-97b374162f1c" (UID: "22626496-4847-4999-a9b4-97b374162f1c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:20:18.818462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:18.818443    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22626496-4847-4999-a9b4-97b374162f1c-kube-api-access-k2phd" pod "22626496-4847-4999-a9b4-97b374162f1c" (UID: "22626496-4847-4999-a9b4-97b374162f1c"). InnerVolumeSpecName "kube-api-access-k2phd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:18.818719 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:18.818701    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22626496-4847-4999-a9b4-97b374162f1c-metrics-certs" pod "22626496-4847-4999-a9b4-97b374162f1c" (UID: "22626496-4847-4999-a9b4-97b374162f1c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:20:18.917312 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:18.917275    4544 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22626496-4847-4999-a9b4-97b374162f1c-openshift-service-ca\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:18.917312 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:18.917305    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2phd\" (UniqueName: \"kubernetes.io/projected/22626496-4847-4999-a9b4-97b374162f1c-kube-api-access-k2phd\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:18.917312 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:18.917317    4544 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22626496-4847-4999-a9b4-97b374162f1c-maas-api-tls\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:18.917533 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:18.917325    4544 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22626496-4847-4999-a9b4-97b374162f1c-metrics-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:19.263128 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:19.263037    4544 generic.go:356] "Generic (PLEG): container finished" podID="22626496-4847-4999-a9b4-97b374162f1c" containerID="7b9cc64ef8e1bafbcba74ae631df78b1d049d08fd3f4e637825ea5f85074772e" exitCode=0
Oct 30 19:20:19.263128 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:19.263088    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k" event={"ID":"22626496-4847-4999-a9b4-97b374162f1c","Type":"ContainerDied","Data":"7b9cc64ef8e1bafbcba74ae631df78b1d049d08fd3f4e637825ea5f85074772e"}
Oct 30 19:20:19.263128 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:19.263126    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k" event={"ID":"22626496-4847-4999-a9b4-97b374162f1c","Type":"ContainerDied","Data":"3d1e0926f1ee6a0e31c64d61e01b32d0acee50dc91f084f8ba15420b71b261c9"}
Oct 30 19:20:19.263426 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:19.263149    4544 scope.go:122] "RemoveContainer" containerID="7b9cc64ef8e1bafbcba74ae631df78b1d049d08fd3f4e637825ea5f85074772e"
Oct 30 19:20:19.263426 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:19.263090    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"
Oct 30 19:20:19.273183 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:19.273168    4544 scope.go:122] "RemoveContainer" containerID="7b9cc64ef8e1bafbcba74ae631df78b1d049d08fd3f4e637825ea5f85074772e"
Oct 30 19:20:19.273434 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:20:19.273416    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b9cc64ef8e1bafbcba74ae631df78b1d049d08fd3f4e637825ea5f85074772e\": container with ID starting with 7b9cc64ef8e1bafbcba74ae631df78b1d049d08fd3f4e637825ea5f85074772e not found: ID does not exist" containerID="7b9cc64ef8e1bafbcba74ae631df78b1d049d08fd3f4e637825ea5f85074772e"
Oct 30 19:20:19.273490 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:19.273443    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b9cc64ef8e1bafbcba74ae631df78b1d049d08fd3f4e637825ea5f85074772e"} err="failed to get container status \"7b9cc64ef8e1bafbcba74ae631df78b1d049d08fd3f4e637825ea5f85074772e\": rpc error: code = NotFound desc = could not find container \"7b9cc64ef8e1bafbcba74ae631df78b1d049d08fd3f4e637825ea5f85074772e\": container with ID starting with 7b9cc64ef8e1bafbcba74ae631df78b1d049d08fd3f4e637825ea5f85074772e not found: ID does not exist"
Oct 30 19:20:19.417457 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:19.417427    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"]
Oct 30 19:20:19.429302 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:19.429261    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f75d5f-645f5c988-lwb6k"]
Oct 30 19:20:20.601086 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:20.601057    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22626496-4847-4999-a9b4-97b374162f1c" path="/var/lib/kubelet/pods/22626496-4847-4999-a9b4-97b374162f1c/volumes"
Oct 30 19:20:23.861368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:23.861334    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq" podUID="8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:20:27.307009 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:27.306964    4544 generic.go:356] "Generic (PLEG): container finished" podID="8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3" containerID="2beb9dd333097b62ffa5094802ee984b37f78e57b87df22d8fdaf841f8481d89" exitCode=0
Oct 30 19:20:27.307379 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:27.307026    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq" event={"ID":"8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3","Type":"ContainerDied","Data":"2beb9dd333097b62ffa5094802ee984b37f78e57b87df22d8fdaf841f8481d89"}
Oct 30 19:20:27.379911 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:27.377876    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq"
Oct 30 19:20:27.502463 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:27.502417    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3-plugins-config-volume\") pod \"8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3\" (UID: \"8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3\") "
Oct 30 19:20:27.503097 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:27.502578    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3-kube-api-access-rrhzq\" (UniqueName: \"kubernetes.io/projected/8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3-kube-api-access-rrhzq\") pod \"8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3\" (UID: \"8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3\") "
Oct 30 19:20:27.503097 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:27.502988    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3-plugins-config-volume" pod "8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3" (UID: "8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:27.505934 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:27.505494    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3-kube-api-access-rrhzq" pod "8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3" (UID: "8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3"). InnerVolumeSpecName "kube-api-access-rrhzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:27.604302 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:27.604275    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:27.604302 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:27.604300    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrhzq\" (UniqueName: \"kubernetes.io/projected/8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3-kube-api-access-rrhzq\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:28.312737 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:28.312705    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq" event={"ID":"8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3","Type":"ContainerDied","Data":"1beac4d83aeffd4938b67ff00f1019b4015a763c40fa779f46f34342d240462c"}
Oct 30 19:20:28.312737 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:28.312731    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq"
Oct 30 19:20:28.313255 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:28.312758    4544 scope.go:122] "RemoveContainer" containerID="2beb9dd333097b62ffa5094802ee984b37f78e57b87df22d8fdaf841f8481d89"
Oct 30 19:20:28.471288 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:28.471257    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq"]
Oct 30 19:20:28.516979 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:28.516948    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f75d5f-f7c697b99-lm5pq"]
Oct 30 19:20:28.601341 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:28.601271    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3" path="/var/lib/kubelet/pods/8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3/volumes"
Oct 30 19:20:35.530545 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:20:35.530506    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:20:39.783233 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:39.783196    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls"]
Oct 30 19:20:39.882392 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:39.882358    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt"]
Oct 30 19:20:44.783699 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:44.783662    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls" podUID="a90f74a2-102e-4495-acc5-069e44aa693d" containerName="payload-processing" containerID="cri-o://2b8e1be8fb00586b851e1132ef9a1e6b156eb9448a5fbbb5e7e9a7645c519b88" gracePeriod=25
Oct 30 19:20:44.883178 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:44.883129    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt" podUID="aed6caf3-296f-46b3-af0d-fd8712434652" containerName="payload-pre-processing" containerID="cri-o://4fe38872d0cad6c92faef4baa00cc91b8348c5a1e53c84c9c5ebb4eb4a0b9e60" gracePeriod=25
Oct 30 19:20:45.037420 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:45.037358    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls"
Oct 30 19:20:45.180409 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:45.180376    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a90f74a2-102e-4495-acc5-069e44aa693d-kube-api-access-mrhxv\" (UniqueName: \"kubernetes.io/projected/a90f74a2-102e-4495-acc5-069e44aa693d-kube-api-access-mrhxv\") pod \"a90f74a2-102e-4495-acc5-069e44aa693d\" (UID: \"a90f74a2-102e-4495-acc5-069e44aa693d\") "
Oct 30 19:20:45.180409 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:45.180414    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a90f74a2-102e-4495-acc5-069e44aa693d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a90f74a2-102e-4495-acc5-069e44aa693d-plugins-config-volume\") pod \"a90f74a2-102e-4495-acc5-069e44aa693d\" (UID: \"a90f74a2-102e-4495-acc5-069e44aa693d\") "
Oct 30 19:20:45.180848 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:45.180824    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a90f74a2-102e-4495-acc5-069e44aa693d-plugins-config-volume" pod "a90f74a2-102e-4495-acc5-069e44aa693d" (UID: "a90f74a2-102e-4495-acc5-069e44aa693d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:45.182521 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:45.182502    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a90f74a2-102e-4495-acc5-069e44aa693d-kube-api-access-mrhxv" pod "a90f74a2-102e-4495-acc5-069e44aa693d" (UID: "a90f74a2-102e-4495-acc5-069e44aa693d"). InnerVolumeSpecName "kube-api-access-mrhxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:45.281790 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:45.281760    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrhxv\" (UniqueName: \"kubernetes.io/projected/a90f74a2-102e-4495-acc5-069e44aa693d-kube-api-access-mrhxv\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:45.281790 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:45.281787    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a90f74a2-102e-4495-acc5-069e44aa693d-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:45.394712 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:45.394636    4544 generic.go:356] "Generic (PLEG): container finished" podID="a90f74a2-102e-4495-acc5-069e44aa693d" containerID="2b8e1be8fb00586b851e1132ef9a1e6b156eb9448a5fbbb5e7e9a7645c519b88" exitCode=0
Oct 30 19:20:45.394712 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:45.394702    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls"
Oct 30 19:20:45.394857 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:45.394724    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls" event={"ID":"a90f74a2-102e-4495-acc5-069e44aa693d","Type":"ContainerDied","Data":"2b8e1be8fb00586b851e1132ef9a1e6b156eb9448a5fbbb5e7e9a7645c519b88"}
Oct 30 19:20:45.394857 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:45.394805    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls" event={"ID":"a90f74a2-102e-4495-acc5-069e44aa693d","Type":"ContainerDied","Data":"7a81c105f12af329c5c55a52d2f041ab91ebb61f0392bb04e85bec039bb99343"}
Oct 30 19:20:45.394857 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:45.394828    4544 scope.go:122] "RemoveContainer" containerID="2b8e1be8fb00586b851e1132ef9a1e6b156eb9448a5fbbb5e7e9a7645c519b88"
Oct 30 19:20:45.404086 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:45.404069    4544 scope.go:122] "RemoveContainer" containerID="2b8e1be8fb00586b851e1132ef9a1e6b156eb9448a5fbbb5e7e9a7645c519b88"
Oct 30 19:20:45.404337 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:20:45.404318    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b8e1be8fb00586b851e1132ef9a1e6b156eb9448a5fbbb5e7e9a7645c519b88\": container with ID starting with 2b8e1be8fb00586b851e1132ef9a1e6b156eb9448a5fbbb5e7e9a7645c519b88 not found: ID does not exist" containerID="2b8e1be8fb00586b851e1132ef9a1e6b156eb9448a5fbbb5e7e9a7645c519b88"
Oct 30 19:20:45.404392 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:45.404346    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b8e1be8fb00586b851e1132ef9a1e6b156eb9448a5fbbb5e7e9a7645c519b88"} err="failed to get container status \"2b8e1be8fb00586b851e1132ef9a1e6b156eb9448a5fbbb5e7e9a7645c519b88\": rpc error: code = NotFound desc = could not find container \"2b8e1be8fb00586b851e1132ef9a1e6b156eb9448a5fbbb5e7e9a7645c519b88\": container with ID starting with 2b8e1be8fb00586b851e1132ef9a1e6b156eb9448a5fbbb5e7e9a7645c519b88 not found: ID does not exist"
Oct 30 19:20:45.553204 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:45.553175    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls"]
Oct 30 19:20:45.558068 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:45.558043    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-29de7b-69df84fc8d-nkzls"]
Oct 30 19:20:46.601867 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:46.601835    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a90f74a2-102e-4495-acc5-069e44aa693d" path="/var/lib/kubelet/pods/a90f74a2-102e-4495-acc5-069e44aa693d/volumes"
Oct 30 19:20:51.484489 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:51.484451    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt" podUID="aed6caf3-296f-46b3-af0d-fd8712434652" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:20:57.730411 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:57.730318    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"]
Oct 30 19:20:57.730924 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:57.730555    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6" podUID="fe8e5ac9-0b0a-4a19-868e-47968373cb8d" containerName="discovery" containerID="cri-o://711fe82075a839a6e7634345f40d6d5bca81a9315a3d737d0d62eb263894923c" gracePeriod=30
Oct 30 19:20:57.872388 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:57.872354    4544 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-776l6 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.84:8080/ready\": dial tcp 10.133.0.84:8080: connect: connection refused" start-of-body=
Oct 30 19:20:57.872550 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:57.872407    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6" podUID="fe8e5ac9-0b0a-4a19-868e-47968373cb8d" containerName="discovery" probeResult="failure" output="Get \"http://10.133.0.84:8080/ready\": dial tcp 10.133.0.84:8080: connect: connection refused"
Oct 30 19:20:57.988107 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:57.988054    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:20:58.001747 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.001722    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-csr-ca-configmap\") pod \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") "
Oct 30 19:20:58.001871 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.001812    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-kubeconfig\") pod \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") "
Oct 30 19:20:58.001871 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.001844    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-token\" (UniqueName: \"kubernetes.io/projected/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-token\") pod \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") "
Oct 30 19:20:58.001978 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.001880    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-cacerts\" (UniqueName: \"kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-cacerts\") pod \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") "
Oct 30 19:20:58.001978 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.001908    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-csr-dns-cert\") pod \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") "
Oct 30 19:20:58.002083 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.001980    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-local-certs\") pod \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") "
Oct 30 19:20:58.002083 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.002046    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-kube-api-access-c82dp\" (UniqueName: \"kubernetes.io/projected/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-kube-api-access-c82dp\") pod \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\" (UID: \"fe8e5ac9-0b0a-4a19-868e-47968373cb8d\") "
Oct 30 19:20:58.002196 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.002088    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-csr-ca-configmap" pod "fe8e5ac9-0b0a-4a19-868e-47968373cb8d" (UID: "fe8e5ac9-0b0a-4a19-868e-47968373cb8d"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:58.002477 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.002444    4544 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-csr-ca-configmap\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:58.004179 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.004149    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-local-certs" pod "fe8e5ac9-0b0a-4a19-868e-47968373cb8d" (UID: "fe8e5ac9-0b0a-4a19-868e-47968373cb8d"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:58.004270 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.004220    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-kube-api-access-c82dp" pod "fe8e5ac9-0b0a-4a19-868e-47968373cb8d" (UID: "fe8e5ac9-0b0a-4a19-868e-47968373cb8d"). InnerVolumeSpecName "kube-api-access-c82dp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:58.004401 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.004376    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-kubeconfig" pod "fe8e5ac9-0b0a-4a19-868e-47968373cb8d" (UID: "fe8e5ac9-0b0a-4a19-868e-47968373cb8d"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:20:58.004546 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.004521    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-token" pod "fe8e5ac9-0b0a-4a19-868e-47968373cb8d" (UID: "fe8e5ac9-0b0a-4a19-868e-47968373cb8d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:58.004546 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.004535    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-csr-dns-cert" pod "fe8e5ac9-0b0a-4a19-868e-47968373cb8d" (UID: "fe8e5ac9-0b0a-4a19-868e-47968373cb8d"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:20:58.004700 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.004558    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-cacerts" pod "fe8e5ac9-0b0a-4a19-868e-47968373cb8d" (UID: "fe8e5ac9-0b0a-4a19-868e-47968373cb8d"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:20:58.104069 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.104028    4544 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-local-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:58.104213 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.104074    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c82dp\" (UniqueName: \"kubernetes.io/projected/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-kube-api-access-c82dp\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:58.104213 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.104089    4544 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-kubeconfig\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:58.104213 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.104103    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:58.104213 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.104116    4544 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-cacerts\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:58.104213 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.104130    4544 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fe8e5ac9-0b0a-4a19-868e-47968373cb8d-istio-csr-dns-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:58.460229 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.460176    4544 generic.go:356] "Generic (PLEG): container finished" podID="fe8e5ac9-0b0a-4a19-868e-47968373cb8d" containerID="711fe82075a839a6e7634345f40d6d5bca81a9315a3d737d0d62eb263894923c" exitCode=0
Oct 30 19:20:58.460229 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.460234    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"
Oct 30 19:20:58.460555 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.460258    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6" event={"ID":"fe8e5ac9-0b0a-4a19-868e-47968373cb8d","Type":"ContainerDied","Data":"711fe82075a839a6e7634345f40d6d5bca81a9315a3d737d0d62eb263894923c"}
Oct 30 19:20:58.460555 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.460323    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6" event={"ID":"fe8e5ac9-0b0a-4a19-868e-47968373cb8d","Type":"ContainerDied","Data":"cb04a172115ffe5554d701e87bcebd292018890bf6844308e1efaa95599bb166"}
Oct 30 19:20:58.460555 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.460356    4544 scope.go:122] "RemoveContainer" containerID="711fe82075a839a6e7634345f40d6d5bca81a9315a3d737d0d62eb263894923c"
Oct 30 19:20:58.479663 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.479635    4544 scope.go:122] "RemoveContainer" containerID="711fe82075a839a6e7634345f40d6d5bca81a9315a3d737d0d62eb263894923c"
Oct 30 19:20:58.480005 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:20:58.479976    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"711fe82075a839a6e7634345f40d6d5bca81a9315a3d737d0d62eb263894923c\": container with ID starting with 711fe82075a839a6e7634345f40d6d5bca81a9315a3d737d0d62eb263894923c not found: ID does not exist" containerID="711fe82075a839a6e7634345f40d6d5bca81a9315a3d737d0d62eb263894923c"
Oct 30 19:20:58.480095 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.480009    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"711fe82075a839a6e7634345f40d6d5bca81a9315a3d737d0d62eb263894923c"} err="failed to get container status \"711fe82075a839a6e7634345f40d6d5bca81a9315a3d737d0d62eb263894923c\": rpc error: code = NotFound desc = could not find container \"711fe82075a839a6e7634345f40d6d5bca81a9315a3d737d0d62eb263894923c\": container with ID starting with 711fe82075a839a6e7634345f40d6d5bca81a9315a3d737d0d62eb263894923c not found: ID does not exist"
Oct 30 19:20:58.623568 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.623528    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"]
Oct 30 19:20:58.669498 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.669464    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-776l6"]
Oct 30 19:20:58.765997 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:58.765972    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"]
Oct 30 19:20:59.466275 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:20:59.466241    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8" podUID="90685ad5-9312-489e-ab58-84c12ebcc5ab" containerName="istio-proxy" containerID="cri-o://c88eb83111408120dbbe38b4f735285b0cead33afc72e1c5248298739a44ed37" gracePeriod=30
Oct 30 19:21:00.600732 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:00.600685    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe8e5ac9-0b0a-4a19-868e-47968373cb8d" path="/var/lib/kubelet/pods/fe8e5ac9-0b0a-4a19-868e-47968373cb8d/volumes"
Oct 30 19:21:01.484385 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:01.484351    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt" podUID="aed6caf3-296f-46b3-af0d-fd8712434652" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:21:04.278677 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.278642    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"]
Oct 30 19:21:04.375060 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.375025    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"]
Oct 30 19:21:04.717750 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.717718    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:21:04.762958 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.762926    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-credential-socket\") pod \"90685ad5-9312-489e-ab58-84c12ebcc5ab\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") "
Oct 30 19:21:04.763113 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.762991    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-podinfo\") pod \"90685ad5-9312-489e-ab58-84c12ebcc5ab\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") "
Oct 30 19:21:04.763184 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.763141    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-token\" (UniqueName: \"kubernetes.io/projected/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-token\") pod \"90685ad5-9312-489e-ab58-84c12ebcc5ab\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") "
Oct 30 19:21:04.763184 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.763169    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-envoy\") pod \"90685ad5-9312-489e-ab58-84c12ebcc5ab\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") "
Oct 30 19:21:04.763184 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.763174    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-credential-socket" pod "90685ad5-9312-489e-ab58-84c12ebcc5ab" (UID: "90685ad5-9312-489e-ab58-84c12ebcc5ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:04.763358 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.763208    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90685ad5-9312-489e-ab58-84c12ebcc5ab-kube-api-access-752cf\" (UniqueName: \"kubernetes.io/projected/90685ad5-9312-489e-ab58-84c12ebcc5ab-kube-api-access-752cf\") pod \"90685ad5-9312-489e-ab58-84c12ebcc5ab\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") "
Oct 30 19:21:04.763358 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.763236    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/90685ad5-9312-489e-ab58-84c12ebcc5ab-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90685ad5-9312-489e-ab58-84c12ebcc5ab-istiod-ca-cert\") pod \"90685ad5-9312-489e-ab58-84c12ebcc5ab\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") "
Oct 30 19:21:04.763358 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.763253    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-workload-socket\") pod \"90685ad5-9312-489e-ab58-84c12ebcc5ab\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") "
Oct 30 19:21:04.763358 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.763277    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-workload-certs\") pod \"90685ad5-9312-489e-ab58-84c12ebcc5ab\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") "
Oct 30 19:21:04.763358 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.763299    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-data\") pod \"90685ad5-9312-489e-ab58-84c12ebcc5ab\" (UID: \"90685ad5-9312-489e-ab58-84c12ebcc5ab\") "
Oct 30 19:21:04.763714 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.763692    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-workload-certs" pod "90685ad5-9312-489e-ab58-84c12ebcc5ab" (UID: "90685ad5-9312-489e-ab58-84c12ebcc5ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:04.763781 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.763715    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:04.763781 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.763732    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90685ad5-9312-489e-ab58-84c12ebcc5ab-istiod-ca-cert" pod "90685ad5-9312-489e-ab58-84c12ebcc5ab" (UID: "90685ad5-9312-489e-ab58-84c12ebcc5ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:04.763892 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.763810    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-workload-socket" pod "90685ad5-9312-489e-ab58-84c12ebcc5ab" (UID: "90685ad5-9312-489e-ab58-84c12ebcc5ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:04.763950 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.763908    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-data" pod "90685ad5-9312-489e-ab58-84c12ebcc5ab" (UID: "90685ad5-9312-489e-ab58-84c12ebcc5ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:04.765376 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.765352    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-podinfo" pod "90685ad5-9312-489e-ab58-84c12ebcc5ab" (UID: "90685ad5-9312-489e-ab58-84c12ebcc5ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:21:04.765493 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.765421    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-token" pod "90685ad5-9312-489e-ab58-84c12ebcc5ab" (UID: "90685ad5-9312-489e-ab58-84c12ebcc5ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:04.765664 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.765648    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-envoy" pod "90685ad5-9312-489e-ab58-84c12ebcc5ab" (UID: "90685ad5-9312-489e-ab58-84c12ebcc5ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:04.765760 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.765745    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90685ad5-9312-489e-ab58-84c12ebcc5ab-kube-api-access-752cf" pod "90685ad5-9312-489e-ab58-84c12ebcc5ab" (UID: "90685ad5-9312-489e-ab58-84c12ebcc5ab"). InnerVolumeSpecName "kube-api-access-752cf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:04.864377 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.864305    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:04.864377 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.864332    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:04.864377 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.864343    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-752cf\" (UniqueName: \"kubernetes.io/projected/90685ad5-9312-489e-ab58-84c12ebcc5ab-kube-api-access-752cf\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:04.864377 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.864352    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90685ad5-9312-489e-ab58-84c12ebcc5ab-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:04.864377 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.864362    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:04.864377 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.864369    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:04.864377 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.864378    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:04.864820 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:04.864386    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90685ad5-9312-489e-ab58-84c12ebcc5ab-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:05.123086 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.123061    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt"
Oct 30 19:21:05.167696 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.167670    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aed6caf3-296f-46b3-af0d-fd8712434652-kube-api-access-hsxdp\" (UniqueName: \"kubernetes.io/projected/aed6caf3-296f-46b3-af0d-fd8712434652-kube-api-access-hsxdp\") pod \"aed6caf3-296f-46b3-af0d-fd8712434652\" (UID: \"aed6caf3-296f-46b3-af0d-fd8712434652\") "
Oct 30 19:21:05.167838 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.167798    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aed6caf3-296f-46b3-af0d-fd8712434652-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aed6caf3-296f-46b3-af0d-fd8712434652-plugins-config-volume\") pod \"aed6caf3-296f-46b3-af0d-fd8712434652\" (UID: \"aed6caf3-296f-46b3-af0d-fd8712434652\") "
Oct 30 19:21:05.168162 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.168140    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aed6caf3-296f-46b3-af0d-fd8712434652-plugins-config-volume" pod "aed6caf3-296f-46b3-af0d-fd8712434652" (UID: "aed6caf3-296f-46b3-af0d-fd8712434652"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:05.169674 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.169652    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aed6caf3-296f-46b3-af0d-fd8712434652-kube-api-access-hsxdp" pod "aed6caf3-296f-46b3-af0d-fd8712434652" (UID: "aed6caf3-296f-46b3-af0d-fd8712434652"). InnerVolumeSpecName "kube-api-access-hsxdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:05.269569 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.269538    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aed6caf3-296f-46b3-af0d-fd8712434652-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:05.269569 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.269566    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsxdp\" (UniqueName: \"kubernetes.io/projected/aed6caf3-296f-46b3-af0d-fd8712434652-kube-api-access-hsxdp\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:05.506943 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.506889    4544 generic.go:356] "Generic (PLEG): container finished" podID="90685ad5-9312-489e-ab58-84c12ebcc5ab" containerID="c88eb83111408120dbbe38b4f735285b0cead33afc72e1c5248298739a44ed37" exitCode=0
Oct 30 19:21:05.507430 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.506955    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"
Oct 30 19:21:05.507430 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.506979    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8" event={"ID":"90685ad5-9312-489e-ab58-84c12ebcc5ab","Type":"ContainerDied","Data":"c88eb83111408120dbbe38b4f735285b0cead33afc72e1c5248298739a44ed37"}
Oct 30 19:21:05.507430 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.507054    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8" event={"ID":"90685ad5-9312-489e-ab58-84c12ebcc5ab","Type":"ContainerDied","Data":"07349a1975ba09a8e14f97bb95ba62415a2f982fe196ddc15a7cf9994d2260dd"}
Oct 30 19:21:05.507430 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.507087    4544 scope.go:122] "RemoveContainer" containerID="c88eb83111408120dbbe38b4f735285b0cead33afc72e1c5248298739a44ed37"
Oct 30 19:21:05.509326 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.509208    4544 generic.go:356] "Generic (PLEG): container finished" podID="aed6caf3-296f-46b3-af0d-fd8712434652" containerID="4fe38872d0cad6c92faef4baa00cc91b8348c5a1e53c84c9c5ebb4eb4a0b9e60" exitCode=0
Oct 30 19:21:05.509326 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.509274    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt" event={"ID":"aed6caf3-296f-46b3-af0d-fd8712434652","Type":"ContainerDied","Data":"4fe38872d0cad6c92faef4baa00cc91b8348c5a1e53c84c9c5ebb4eb4a0b9e60"}
Oct 30 19:21:05.509455 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.509329    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt" event={"ID":"aed6caf3-296f-46b3-af0d-fd8712434652","Type":"ContainerDied","Data":"19f573785003f2a5c2a5a5593084914b12206801897b80665e2309cf0cf8d554"}
Oct 30 19:21:05.509455 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.509418    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt"
Oct 30 19:21:05.521735 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.519976    4544 scope.go:122] "RemoveContainer" containerID="1541c711d30aa1903b3b2c837e2cf0a78a2402858490f4f01a9c25dd97ea34ec"
Oct 30 19:21:05.529289 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.529260    4544 scope.go:122] "RemoveContainer" containerID="c88eb83111408120dbbe38b4f735285b0cead33afc72e1c5248298739a44ed37"
Oct 30 19:21:05.529667 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:21:05.529646    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c88eb83111408120dbbe38b4f735285b0cead33afc72e1c5248298739a44ed37\": container with ID starting with c88eb83111408120dbbe38b4f735285b0cead33afc72e1c5248298739a44ed37 not found: ID does not exist" containerID="c88eb83111408120dbbe38b4f735285b0cead33afc72e1c5248298739a44ed37"
Oct 30 19:21:05.529737 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.529677    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c88eb83111408120dbbe38b4f735285b0cead33afc72e1c5248298739a44ed37"} err="failed to get container status \"c88eb83111408120dbbe38b4f735285b0cead33afc72e1c5248298739a44ed37\": rpc error: code = NotFound desc = could not find container \"c88eb83111408120dbbe38b4f735285b0cead33afc72e1c5248298739a44ed37\": container with ID starting with c88eb83111408120dbbe38b4f735285b0cead33afc72e1c5248298739a44ed37 not found: ID does not exist"
Oct 30 19:21:05.529737 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.529696    4544 scope.go:122] "RemoveContainer" containerID="1541c711d30aa1903b3b2c837e2cf0a78a2402858490f4f01a9c25dd97ea34ec"
Oct 30 19:21:05.530013 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:21:05.529997    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1541c711d30aa1903b3b2c837e2cf0a78a2402858490f4f01a9c25dd97ea34ec\": container with ID starting with 1541c711d30aa1903b3b2c837e2cf0a78a2402858490f4f01a9c25dd97ea34ec not found: ID does not exist" containerID="1541c711d30aa1903b3b2c837e2cf0a78a2402858490f4f01a9c25dd97ea34ec"
Oct 30 19:21:05.530079 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.530016    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1541c711d30aa1903b3b2c837e2cf0a78a2402858490f4f01a9c25dd97ea34ec"} err="failed to get container status \"1541c711d30aa1903b3b2c837e2cf0a78a2402858490f4f01a9c25dd97ea34ec\": rpc error: code = NotFound desc = could not find container \"1541c711d30aa1903b3b2c837e2cf0a78a2402858490f4f01a9c25dd97ea34ec\": container with ID starting with 1541c711d30aa1903b3b2c837e2cf0a78a2402858490f4f01a9c25dd97ea34ec not found: ID does not exist"
Oct 30 19:21:05.530079 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.530031    4544 scope.go:122] "RemoveContainer" containerID="4fe38872d0cad6c92faef4baa00cc91b8348c5a1e53c84c9c5ebb4eb4a0b9e60"
Oct 30 19:21:05.538976 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:21:05.538933    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:21:05.539379 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.539364    4544 scope.go:122] "RemoveContainer" containerID="4fe38872d0cad6c92faef4baa00cc91b8348c5a1e53c84c9c5ebb4eb4a0b9e60"
Oct 30 19:21:05.539670 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:21:05.539646    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fe38872d0cad6c92faef4baa00cc91b8348c5a1e53c84c9c5ebb4eb4a0b9e60\": container with ID starting with 4fe38872d0cad6c92faef4baa00cc91b8348c5a1e53c84c9c5ebb4eb4a0b9e60 not found: ID does not exist" containerID="4fe38872d0cad6c92faef4baa00cc91b8348c5a1e53c84c9c5ebb4eb4a0b9e60"
Oct 30 19:21:05.539737 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.539681    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fe38872d0cad6c92faef4baa00cc91b8348c5a1e53c84c9c5ebb4eb4a0b9e60"} err="failed to get container status \"4fe38872d0cad6c92faef4baa00cc91b8348c5a1e53c84c9c5ebb4eb4a0b9e60\": rpc error: code = NotFound desc = could not find container \"4fe38872d0cad6c92faef4baa00cc91b8348c5a1e53c84c9c5ebb4eb4a0b9e60\": container with ID starting with 4fe38872d0cad6c92faef4baa00cc91b8348c5a1e53c84c9c5ebb4eb4a0b9e60 not found: ID does not exist"
Oct 30 19:21:05.663567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.663530    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"]
Oct 30 19:21:05.708834 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.708804    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-29de7b-openshift-default-6cdb8dfcdc-d6ws8"]
Oct 30 19:21:05.771398 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.771364    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt"]
Oct 30 19:21:05.774507 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:05.774484    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-29de7b-5485c658d7-2nzbt"]
Oct 30 19:21:06.601344 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:06.601313    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="90685ad5-9312-489e-ab58-84c12ebcc5ab" path="/var/lib/kubelet/pods/90685ad5-9312-489e-ab58-84c12ebcc5ab/volumes"
Oct 30 19:21:06.601840 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:06.601720    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aed6caf3-296f-46b3-af0d-fd8712434652" path="/var/lib/kubelet/pods/aed6caf3-296f-46b3-af0d-fd8712434652/volumes"
Oct 30 19:21:19.298091 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.298045    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main" containerID="cri-o://e25da3c1ffdc4287c16b1eae1abd6846f72686e98839a185866d7e601bf67620" gracePeriod=45
Oct 30 19:21:19.394232 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.394192    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main" containerID="cri-o://04a75aa9862defc69ec19016034737f0d3834e74221a54eb91d9a56fbd3f4a1e" gracePeriod=45
Oct 30 19:21:19.579555 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.579519    4544 generic.go:356] "Generic (PLEG): container finished" podID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerID="e25da3c1ffdc4287c16b1eae1abd6846f72686e98839a185866d7e601bf67620" exitCode=0
Oct 30 19:21:19.579844 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.579624    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" event={"ID":"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9","Type":"ContainerDied","Data":"e25da3c1ffdc4287c16b1eae1abd6846f72686e98839a185866d7e601bf67620"}
Oct 30 19:21:19.581245 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.581211    4544 generic.go:356] "Generic (PLEG): container finished" podID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerID="04a75aa9862defc69ec19016034737f0d3834e74221a54eb91d9a56fbd3f4a1e" exitCode=0
Oct 30 19:21:19.581376 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.581282    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" event={"ID":"12f140d3-7cea-4dc9-8a60-e8d11acca08f","Type":"ContainerDied","Data":"04a75aa9862defc69ec19016034737f0d3834e74221a54eb91d9a56fbd3f4a1e"}
Oct 30 19:21:19.641071 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.641047    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 19:21:19.669123 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.669092    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 19:21:19.715088 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.715053    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-tmp-dir\") pod \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") "
Oct 30 19:21:19.715220 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.715118    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-dshm\") pod \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") "
Oct 30 19:21:19.715220 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.715180    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-model-cache\") pod \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") "
Oct 30 19:21:19.715359 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.715221    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-kube-api-access-wqcvs\" (UniqueName: \"kubernetes.io/projected/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-kube-api-access-wqcvs\") pod \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") "
Oct 30 19:21:19.715359 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.715268    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-home\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-home\") pod \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") "
Oct 30 19:21:19.715359 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.715327    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-tls-certs\" (UniqueName: \"kubernetes.io/secret/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-tls-certs\") pod \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\" (UID: \"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9\") "
Oct 30 19:21:19.715517 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.715324    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-tmp-dir" pod "91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" (UID: "91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:19.715517 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.715491    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-model-cache" pod "91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" (UID: "91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:19.715517 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.715485    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-home" pod "91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" (UID: "91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:19.715705 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.715684    4544 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-model-cache\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:19.715764 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.715704    4544 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-home\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:19.715764 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.715716    4544 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-tmp-dir\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:19.717405 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.717377    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-dshm" pod "91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" (UID: "91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:19.717501 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.717449    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-kube-api-access-wqcvs" pod "91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" (UID: "91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9"). InnerVolumeSpecName "kube-api-access-wqcvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:19.717501 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.717470    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-tls-certs" pod "91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" (UID: "91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:19.817105 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.817028    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12f140d3-7cea-4dc9-8a60-e8d11acca08f-kube-api-access-frltf\" (UniqueName: \"kubernetes.io/projected/12f140d3-7cea-4dc9-8a60-e8d11acca08f-kube-api-access-frltf\") pod \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") "
Oct 30 19:21:19.817105 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.817075    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-tmp-dir\") pod \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") "
Oct 30 19:21:19.817105 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.817098    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-model-cache\") pod \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") "
Oct 30 19:21:19.817388 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.817127    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-dshm\") pod \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") "
Oct 30 19:21:19.817388 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.817157    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/12f140d3-7cea-4dc9-8a60-e8d11acca08f-tls-certs\" (UniqueName: \"kubernetes.io/secret/12f140d3-7cea-4dc9-8a60-e8d11acca08f-tls-certs\") pod \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") "
Oct 30 19:21:19.817388 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.817248    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-home\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-home\") pod \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\" (UID: \"12f140d3-7cea-4dc9-8a60-e8d11acca08f\") "
Oct 30 19:21:19.817555 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.817385    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-tmp-dir" pod "12f140d3-7cea-4dc9-8a60-e8d11acca08f" (UID: "12f140d3-7cea-4dc9-8a60-e8d11acca08f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:19.817555 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.817477    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-model-cache" pod "12f140d3-7cea-4dc9-8a60-e8d11acca08f" (UID: "12f140d3-7cea-4dc9-8a60-e8d11acca08f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:19.817555 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.817509    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqcvs\" (UniqueName: \"kubernetes.io/projected/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-kube-api-access-wqcvs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:19.817555 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.817529    4544 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-tls-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:19.817555 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.817538    4544 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9-dshm\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:19.817555 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.817546    4544 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-tmp-dir\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:19.817896 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.817573    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-home" pod "12f140d3-7cea-4dc9-8a60-e8d11acca08f" (UID: "12f140d3-7cea-4dc9-8a60-e8d11acca08f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:19.819554 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.819520    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-dshm" pod "12f140d3-7cea-4dc9-8a60-e8d11acca08f" (UID: "12f140d3-7cea-4dc9-8a60-e8d11acca08f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:19.819819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.819798    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12f140d3-7cea-4dc9-8a60-e8d11acca08f-kube-api-access-frltf" pod "12f140d3-7cea-4dc9-8a60-e8d11acca08f" (UID: "12f140d3-7cea-4dc9-8a60-e8d11acca08f"). InnerVolumeSpecName "kube-api-access-frltf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:19.819899 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.819803    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12f140d3-7cea-4dc9-8a60-e8d11acca08f-tls-certs" pod "12f140d3-7cea-4dc9-8a60-e8d11acca08f" (UID: "12f140d3-7cea-4dc9-8a60-e8d11acca08f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:19.918914 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.918876    4544 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-model-cache\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:19.918914 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.918904    4544 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-dshm\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:19.918914 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.918914    4544 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12f140d3-7cea-4dc9-8a60-e8d11acca08f-tls-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:19.918914 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.918923    4544 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12f140d3-7cea-4dc9-8a60-e8d11acca08f-home\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:19.919181 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:19.918933    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frltf\" (UniqueName: \"kubernetes.io/projected/12f140d3-7cea-4dc9-8a60-e8d11acca08f-kube-api-access-frltf\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:20.587456 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:20.587381    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"
Oct 30 19:21:20.587900 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:20.587382    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb" event={"ID":"91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9","Type":"ContainerDied","Data":"941030b1cfd17c1b0fc92503e739dbde9b48d41cc2531143bc2ad88a5f990cc4"}
Oct 30 19:21:20.587900 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:20.587515    4544 scope.go:122] "RemoveContainer" containerID="e25da3c1ffdc4287c16b1eae1abd6846f72686e98839a185866d7e601bf67620"
Oct 30 19:21:20.588935 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:20.588758    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"
Oct 30 19:21:20.588935 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:20.588779    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc" event={"ID":"12f140d3-7cea-4dc9-8a60-e8d11acca08f","Type":"ContainerDied","Data":"af89b5492fe9fd76137714d568c10deaad4c5b41c7f606e1c6eb338dc71946fb"}
Oct 30 19:21:20.597501 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:20.597340    4544 scope.go:122] "RemoveContainer" containerID="04a75aa9862defc69ec19016034737f0d3834e74221a54eb91d9a56fbd3f4a1e"
Oct 30 19:21:20.740895 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:20.740864    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"]
Oct 30 19:21:20.787319 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:20.787291    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-f5705f/simulator-w0-f5705f-kserve-85499df5bd-7xklc"]
Oct 30 19:21:20.848615 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:20.848539    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"]
Oct 30 19:21:20.854908 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:20.854884    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-f5705f/distinct-2-w0-f5705f-kserve-747d77cf-8d4fb"]
Oct 30 19:21:22.601377 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:22.601344    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" path="/var/lib/kubelet/pods/12f140d3-7cea-4dc9-8a60-e8d11acca08f/volumes"
Oct 30 19:21:22.601800 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:22.601761    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" path="/var/lib/kubelet/pods/91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9/volumes"
Oct 30 19:21:25.339490 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.339456    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"]
Oct 30 19:21:25.339904 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.339694    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8" podUID="b7fb9b6d-7af3-4936-9e78-fe761480c080" containerName="maas-api" containerID="cri-o://0501699566a73a921d1af02f88db68cd6dd9de8af12cee4498ab0e8bf17c5bbd" gracePeriod=30
Oct 30 19:21:25.591055 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.590994    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 19:21:25.614354 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.614313    4544 generic.go:356] "Generic (PLEG): container finished" podID="b7fb9b6d-7af3-4936-9e78-fe761480c080" containerID="0501699566a73a921d1af02f88db68cd6dd9de8af12cee4498ab0e8bf17c5bbd" exitCode=0
Oct 30 19:21:25.614526 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.614361    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8" event={"ID":"b7fb9b6d-7af3-4936-9e78-fe761480c080","Type":"ContainerDied","Data":"0501699566a73a921d1af02f88db68cd6dd9de8af12cee4498ab0e8bf17c5bbd"}
Oct 30 19:21:25.614526 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.614400    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8" event={"ID":"b7fb9b6d-7af3-4936-9e78-fe761480c080","Type":"ContainerDied","Data":"0bed570d8fb9a5d991df22b576bbd97368689d31bc2467cd3f285bde8334b063"}
Oct 30 19:21:25.614526 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.614426    4544 scope.go:122] "RemoveContainer" containerID="0501699566a73a921d1af02f88db68cd6dd9de8af12cee4498ab0e8bf17c5bbd"
Oct 30 19:21:25.614526 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.614365    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"
Oct 30 19:21:25.623048 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.623023    4544 scope.go:122] "RemoveContainer" containerID="0501699566a73a921d1af02f88db68cd6dd9de8af12cee4498ab0e8bf17c5bbd"
Oct 30 19:21:25.623327 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:21:25.623306    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0501699566a73a921d1af02f88db68cd6dd9de8af12cee4498ab0e8bf17c5bbd\": container with ID starting with 0501699566a73a921d1af02f88db68cd6dd9de8af12cee4498ab0e8bf17c5bbd not found: ID does not exist" containerID="0501699566a73a921d1af02f88db68cd6dd9de8af12cee4498ab0e8bf17c5bbd"
Oct 30 19:21:25.623414 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.623346    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0501699566a73a921d1af02f88db68cd6dd9de8af12cee4498ab0e8bf17c5bbd"} err="failed to get container status \"0501699566a73a921d1af02f88db68cd6dd9de8af12cee4498ab0e8bf17c5bbd\": rpc error: code = NotFound desc = could not find container \"0501699566a73a921d1af02f88db68cd6dd9de8af12cee4498ab0e8bf17c5bbd\": container with ID starting with 0501699566a73a921d1af02f88db68cd6dd9de8af12cee4498ab0e8bf17c5bbd not found: ID does not exist"
Oct 30 19:21:25.675095 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.675053    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b7fb9b6d-7af3-4936-9e78-fe761480c080-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7fb9b6d-7af3-4936-9e78-fe761480c080-maas-api-tls\") pod \"b7fb9b6d-7af3-4936-9e78-fe761480c080\" (UID: \"b7fb9b6d-7af3-4936-9e78-fe761480c080\") "
Oct 30 19:21:25.675249 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.675147    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b7fb9b6d-7af3-4936-9e78-fe761480c080-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7fb9b6d-7af3-4936-9e78-fe761480c080-openshift-service-ca\") pod \"b7fb9b6d-7af3-4936-9e78-fe761480c080\" (UID: \"b7fb9b6d-7af3-4936-9e78-fe761480c080\") "
Oct 30 19:21:25.675311 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.675253    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b7fb9b6d-7af3-4936-9e78-fe761480c080-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7fb9b6d-7af3-4936-9e78-fe761480c080-metrics-certs\") pod \"b7fb9b6d-7af3-4936-9e78-fe761480c080\" (UID: \"b7fb9b6d-7af3-4936-9e78-fe761480c080\") "
Oct 30 19:21:25.675368 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.675307    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7fb9b6d-7af3-4936-9e78-fe761480c080-kube-api-access-qh2p2\" (UniqueName: \"kubernetes.io/projected/b7fb9b6d-7af3-4936-9e78-fe761480c080-kube-api-access-qh2p2\") pod \"b7fb9b6d-7af3-4936-9e78-fe761480c080\" (UID: \"b7fb9b6d-7af3-4936-9e78-fe761480c080\") "
Oct 30 19:21:25.675529 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.675493    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7fb9b6d-7af3-4936-9e78-fe761480c080-openshift-service-ca" pod "b7fb9b6d-7af3-4936-9e78-fe761480c080" (UID: "b7fb9b6d-7af3-4936-9e78-fe761480c080"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:25.675642 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.675625    4544 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7fb9b6d-7af3-4936-9e78-fe761480c080-openshift-service-ca\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:25.677249 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.677223    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7fb9b6d-7af3-4936-9e78-fe761480c080-maas-api-tls" pod "b7fb9b6d-7af3-4936-9e78-fe761480c080" (UID: "b7fb9b6d-7af3-4936-9e78-fe761480c080"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:25.677401 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.677379    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7fb9b6d-7af3-4936-9e78-fe761480c080-metrics-certs" pod "b7fb9b6d-7af3-4936-9e78-fe761480c080" (UID: "b7fb9b6d-7af3-4936-9e78-fe761480c080"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:25.677467 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.677457    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7fb9b6d-7af3-4936-9e78-fe761480c080-kube-api-access-qh2p2" pod "b7fb9b6d-7af3-4936-9e78-fe761480c080" (UID: "b7fb9b6d-7af3-4936-9e78-fe761480c080"). InnerVolumeSpecName "kube-api-access-qh2p2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:25.776771 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.776735    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh2p2\" (UniqueName: \"kubernetes.io/projected/b7fb9b6d-7af3-4936-9e78-fe761480c080-kube-api-access-qh2p2\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:25.776771 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.776770    4544 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7fb9b6d-7af3-4936-9e78-fe761480c080-maas-api-tls\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:25.777053 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:25.776786    4544 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7fb9b6d-7af3-4936-9e78-fe761480c080-metrics-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:26.028280 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:26.028246    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"]
Oct 30 19:21:26.034574 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:26.034537    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5705f-5d897bbb9-2z6j8"]
Oct 30 19:21:26.601912 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:26.601880    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b7fb9b6d-7af3-4936-9e78-fe761480c080" path="/var/lib/kubelet/pods/b7fb9b6d-7af3-4936-9e78-fe761480c080/volumes"
Oct 30 19:21:35.530476 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:21:35.530440    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:21:41.237146 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.237114    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7496565bcf-dfskg"]
Oct 30 19:21:41.237700 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.237309    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-7496565bcf-dfskg" podUID="659356cf-1afb-42ba-8d0e-d489739f4810" containerName="manager" containerID="cri-o://f4d70156fbb5069ba50d2b8029af923a68a82fb533b04913ecc67974a4d771a3" gracePeriod=30
Oct 30 19:21:41.508021 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.507990    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7496565bcf-dfskg"
Oct 30 19:21:41.624452 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.624418    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/659356cf-1afb-42ba-8d0e-d489739f4810-metrics-certs\" (UniqueName: \"kubernetes.io/secret/659356cf-1afb-42ba-8d0e-d489739f4810-metrics-certs\") pod \"659356cf-1afb-42ba-8d0e-d489739f4810\" (UID: \"659356cf-1afb-42ba-8d0e-d489739f4810\") "
Oct 30 19:21:41.624670 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.624461    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/659356cf-1afb-42ba-8d0e-d489739f4810-kube-api-access-rvxp4\" (UniqueName: \"kubernetes.io/projected/659356cf-1afb-42ba-8d0e-d489739f4810-kube-api-access-rvxp4\") pod \"659356cf-1afb-42ba-8d0e-d489739f4810\" (UID: \"659356cf-1afb-42ba-8d0e-d489739f4810\") "
Oct 30 19:21:41.624670 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.624547    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/659356cf-1afb-42ba-8d0e-d489739f4810-webhook-cert\" (UniqueName: \"kubernetes.io/secret/659356cf-1afb-42ba-8d0e-d489739f4810-webhook-cert\") pod \"659356cf-1afb-42ba-8d0e-d489739f4810\" (UID: \"659356cf-1afb-42ba-8d0e-d489739f4810\") "
Oct 30 19:21:41.627037 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.626993    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/659356cf-1afb-42ba-8d0e-d489739f4810-kube-api-access-rvxp4" pod "659356cf-1afb-42ba-8d0e-d489739f4810" (UID: "659356cf-1afb-42ba-8d0e-d489739f4810"). InnerVolumeSpecName "kube-api-access-rvxp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:41.629764 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.629725    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/659356cf-1afb-42ba-8d0e-d489739f4810-metrics-certs" pod "659356cf-1afb-42ba-8d0e-d489739f4810" (UID: "659356cf-1afb-42ba-8d0e-d489739f4810"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:41.629764 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.629748    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/659356cf-1afb-42ba-8d0e-d489739f4810-webhook-cert" pod "659356cf-1afb-42ba-8d0e-d489739f4810" (UID: "659356cf-1afb-42ba-8d0e-d489739f4810"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:21:41.689437 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.689400    4544 generic.go:356] "Generic (PLEG): container finished" podID="659356cf-1afb-42ba-8d0e-d489739f4810" containerID="f4d70156fbb5069ba50d2b8029af923a68a82fb533b04913ecc67974a4d771a3" exitCode=0
Oct 30 19:21:41.689643 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.689443    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7496565bcf-dfskg"
Oct 30 19:21:41.689643 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.689466    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7496565bcf-dfskg" event={"ID":"659356cf-1afb-42ba-8d0e-d489739f4810","Type":"ContainerDied","Data":"f4d70156fbb5069ba50d2b8029af923a68a82fb533b04913ecc67974a4d771a3"}
Oct 30 19:21:41.689643 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.689530    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7496565bcf-dfskg" event={"ID":"659356cf-1afb-42ba-8d0e-d489739f4810","Type":"ContainerDied","Data":"0488063ff5b630db5b403b3628069855b5a5b2d1fb487317892b71e27760ba64"}
Oct 30 19:21:41.689643 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.689561    4544 scope.go:122] "RemoveContainer" containerID="f4d70156fbb5069ba50d2b8029af923a68a82fb533b04913ecc67974a4d771a3"
Oct 30 19:21:41.700049 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.700029    4544 scope.go:122] "RemoveContainer" containerID="f4d70156fbb5069ba50d2b8029af923a68a82fb533b04913ecc67974a4d771a3"
Oct 30 19:21:41.700333 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:21:41.700311    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4d70156fbb5069ba50d2b8029af923a68a82fb533b04913ecc67974a4d771a3\": container with ID starting with f4d70156fbb5069ba50d2b8029af923a68a82fb533b04913ecc67974a4d771a3 not found: ID does not exist" containerID="f4d70156fbb5069ba50d2b8029af923a68a82fb533b04913ecc67974a4d771a3"
Oct 30 19:21:41.700413 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.700345    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4d70156fbb5069ba50d2b8029af923a68a82fb533b04913ecc67974a4d771a3"} err="failed to get container status \"f4d70156fbb5069ba50d2b8029af923a68a82fb533b04913ecc67974a4d771a3\": rpc error: code = NotFound desc = could not find container \"f4d70156fbb5069ba50d2b8029af923a68a82fb533b04913ecc67974a4d771a3\": container with ID starting with f4d70156fbb5069ba50d2b8029af923a68a82fb533b04913ecc67974a4d771a3 not found: ID does not exist"
Oct 30 19:21:41.725245 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.725218    4544 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/659356cf-1afb-42ba-8d0e-d489739f4810-webhook-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:41.725245 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.725243    4544 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/659356cf-1afb-42ba-8d0e-d489739f4810-metrics-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:41.725245 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.725253    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvxp4\" (UniqueName: \"kubernetes.io/projected/659356cf-1afb-42ba-8d0e-d489739f4810-kube-api-access-rvxp4\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:41.846247 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.846215    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7496565bcf-dfskg"]
Oct 30 19:21:41.853399 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:41.853375    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7496565bcf-dfskg"]
Oct 30 19:21:42.612534 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.612142    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="659356cf-1afb-42ba-8d0e-d489739f4810" path="/var/lib/kubelet/pods/659356cf-1afb-42ba-8d0e-d489739f4810/volumes"
Oct 30 19:21:42.760027 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.759987    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7496565bcf-9jt85"]
Oct 30 19:21:42.761021 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.760758    4544 cpu_manager.go:414] "containerMap: removing container" podUID="659356cf-1afb-42ba-8d0e-d489739f4810" containerName="manager"
Oct 30 19:21:42.761205 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.761187    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="659356cf-1afb-42ba-8d0e-d489739f4810" containerName="manager"
Oct 30 19:21:42.761318 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.761304    4544 cpu_manager.go:414] "containerMap: removing container" podUID="aed6caf3-296f-46b3-af0d-fd8712434652" containerName="payload-pre-processing"
Oct 30 19:21:42.761428 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.761413    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aed6caf3-296f-46b3-af0d-fd8712434652" containerName="payload-pre-processing"
Oct 30 19:21:42.761615 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.761564    4544 cpu_manager.go:414] "containerMap: removing container" podUID="dddf9498-52dc-4712-933a-fee14fc5eca9" containerName="maas-api"
Oct 30 19:21:42.761728 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.761713    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dddf9498-52dc-4712-933a-fee14fc5eca9" containerName="maas-api"
Oct 30 19:21:42.761854 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.761842    4544 cpu_manager.go:414] "containerMap: removing container" podUID="a90f74a2-102e-4495-acc5-069e44aa693d" containerName="payload-processing"
Oct 30 19:21:42.761951 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.761937    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a90f74a2-102e-4495-acc5-069e44aa693d" containerName="payload-processing"
Oct 30 19:21:42.762042 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.762032    4544 cpu_manager.go:414] "containerMap: removing container" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main"
Oct 30 19:21:42.762136 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.762124    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main"
Oct 30 19:21:42.762232 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.762220    4544 cpu_manager.go:414] "containerMap: removing container" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main"
Oct 30 19:21:42.762318 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.762306    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main"
Oct 30 19:21:42.762422 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.762411    4544 cpu_manager.go:414] "containerMap: removing container" podUID="fe8e5ac9-0b0a-4a19-868e-47968373cb8d" containerName="discovery"
Oct 30 19:21:42.762514 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.762501    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe8e5ac9-0b0a-4a19-868e-47968373cb8d" containerName="discovery"
Oct 30 19:21:42.762633 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.762620    4544 cpu_manager.go:414] "containerMap: removing container" podUID="9dd18fca-58d5-45d5-8894-1c12386726fa" containerName="istio-proxy"
Oct 30 19:21:42.762739 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.762727    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9dd18fca-58d5-45d5-8894-1c12386726fa" containerName="istio-proxy"
Oct 30 19:21:42.762837 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.762826    4544 cpu_manager.go:414] "containerMap: removing container" podUID="8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3" containerName="payload-pre-processing"
Oct 30 19:21:42.762930 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.762918    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3" containerName="payload-pre-processing"
Oct 30 19:21:42.763035 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.763023    4544 cpu_manager.go:414] "containerMap: removing container" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main"
Oct 30 19:21:42.763119 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.763108    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main"
Oct 30 19:21:42.763200 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.763190    4544 cpu_manager.go:414] "containerMap: removing container" podUID="90685ad5-9312-489e-ab58-84c12ebcc5ab" containerName="istio-proxy"
Oct 30 19:21:42.763286 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.763275    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90685ad5-9312-489e-ab58-84c12ebcc5ab" containerName="istio-proxy"
Oct 30 19:21:42.763371 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.763361    4544 cpu_manager.go:414] "containerMap: removing container" podUID="90685ad5-9312-489e-ab58-84c12ebcc5ab" containerName="istio-proxy"
Oct 30 19:21:42.763462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.763451    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90685ad5-9312-489e-ab58-84c12ebcc5ab" containerName="istio-proxy"
Oct 30 19:21:42.763561 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.763550    4544 cpu_manager.go:414] "containerMap: removing container" podUID="5662c153-5928-418f-b1fa-74bd0d7a2727" containerName="payload-processing"
Oct 30 19:21:42.763666 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.763653    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5662c153-5928-418f-b1fa-74bd0d7a2727" containerName="payload-processing"
Oct 30 19:21:42.763780 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.763769    4544 cpu_manager.go:414] "containerMap: removing container" podUID="b7fb9b6d-7af3-4936-9e78-fe761480c080" containerName="maas-api"
Oct 30 19:21:42.763871 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.763858    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7fb9b6d-7af3-4936-9e78-fe761480c080" containerName="maas-api"
Oct 30 19:21:42.763967 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.763956    4544 cpu_manager.go:414] "containerMap: removing container" podUID="22626496-4847-4999-a9b4-97b374162f1c" containerName="maas-api"
Oct 30 19:21:42.764053 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.764042    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22626496-4847-4999-a9b4-97b374162f1c" containerName="maas-api"
Oct 30 19:21:42.764153 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.764143    4544 cpu_manager.go:414] "containerMap: removing container" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main"
Oct 30 19:21:42.764231 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.764221    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main"
Oct 30 19:21:42.764536 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.764518    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="9dd18fca-58d5-45d5-8894-1c12386726fa" containerName="istio-proxy"
Oct 30 19:21:42.764696 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.764681    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e9121ca-ec5a-4c12-a0e7-f9b70d561ef3" containerName="payload-pre-processing"
Oct 30 19:21:42.764811 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.764798    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="91f8e092-eb1a-4c0f-9c1c-35195b4c8ed9" containerName="main"
Oct 30 19:21:42.764910 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.764899    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="90685ad5-9312-489e-ab58-84c12ebcc5ab" containerName="istio-proxy"
Oct 30 19:21:42.765004 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.764993    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="90685ad5-9312-489e-ab58-84c12ebcc5ab" containerName="istio-proxy"
Oct 30 19:21:42.776861 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.776837    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="5662c153-5928-418f-b1fa-74bd0d7a2727" containerName="payload-processing"
Oct 30 19:21:42.777048 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.777034    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7fb9b6d-7af3-4936-9e78-fe761480c080" containerName="maas-api"
Oct 30 19:21:42.777254 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.777239    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="22626496-4847-4999-a9b4-97b374162f1c" containerName="maas-api"
Oct 30 19:21:42.777362 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.777350    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="12f140d3-7cea-4dc9-8a60-e8d11acca08f" containerName="main"
Oct 30 19:21:42.777447 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.777437    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="659356cf-1afb-42ba-8d0e-d489739f4810" containerName="manager"
Oct 30 19:21:42.777534 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.777523    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="aed6caf3-296f-46b3-af0d-fd8712434652" containerName="payload-pre-processing"
Oct 30 19:21:42.777703 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.777648    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="dddf9498-52dc-4712-933a-fee14fc5eca9" containerName="maas-api"
Oct 30 19:21:42.777703 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.777702    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="79574f74-b0c4-4584-8f18-84f1659e020b" containerName="main"
Oct 30 19:21:42.777893 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.777712    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="a90f74a2-102e-4495-acc5-069e44aa693d" containerName="payload-processing"
Oct 30 19:21:42.777893 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.777723    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="17c5e93f-ffe3-47c7-9c29-b2bf0e6e9a06" containerName="main"
Oct 30 19:21:42.777893 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.777733    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe8e5ac9-0b0a-4a19-868e-47968373cb8d" containerName="discovery"
Oct 30 19:21:42.782849 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.782822    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7496565bcf-9jt85"
Oct 30 19:21:42.834458 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.834425    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vkhgq\""
Oct 30 19:21:42.834962 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.834932    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a43508c1-d783-41eb-8c54-6dd31e1fafab-metrics-certs\") pod \"maas-controller-7496565bcf-9jt85\" (UID: \"a43508c1-d783-41eb-8c54-6dd31e1fafab\") " pod="opendatahub/maas-controller-7496565bcf-9jt85"
Oct 30 19:21:42.835119 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.835061    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a43508c1-d783-41eb-8c54-6dd31e1fafab-webhook-cert\") pod \"maas-controller-7496565bcf-9jt85\" (UID: \"a43508c1-d783-41eb-8c54-6dd31e1fafab\") " pod="opendatahub/maas-controller-7496565bcf-9jt85"
Oct 30 19:21:42.835119 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.835107    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgqqt\" (UniqueName: \"kubernetes.io/projected/a43508c1-d783-41eb-8c54-6dd31e1fafab-kube-api-access-rgqqt\") pod \"maas-controller-7496565bcf-9jt85\" (UID: \"a43508c1-d783-41eb-8c54-6dd31e1fafab\") " pod="opendatahub/maas-controller-7496565bcf-9jt85"
Oct 30 19:21:42.880721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.880585    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 19:21:42.880903 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.880772    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 19:21:42.880903 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.880857    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 19:21:42.882823 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.882798    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7496565bcf-9jt85"]
Oct 30 19:21:42.936279 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.936242    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a43508c1-d783-41eb-8c54-6dd31e1fafab-webhook-cert\") pod \"maas-controller-7496565bcf-9jt85\" (UID: \"a43508c1-d783-41eb-8c54-6dd31e1fafab\") " pod="opendatahub/maas-controller-7496565bcf-9jt85"
Oct 30 19:21:42.936473 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.936313    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgqqt\" (UniqueName: \"kubernetes.io/projected/a43508c1-d783-41eb-8c54-6dd31e1fafab-kube-api-access-rgqqt\") pod \"maas-controller-7496565bcf-9jt85\" (UID: \"a43508c1-d783-41eb-8c54-6dd31e1fafab\") " pod="opendatahub/maas-controller-7496565bcf-9jt85"
Oct 30 19:21:42.936473 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.936382    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a43508c1-d783-41eb-8c54-6dd31e1fafab-metrics-certs\") pod \"maas-controller-7496565bcf-9jt85\" (UID: \"a43508c1-d783-41eb-8c54-6dd31e1fafab\") " pod="opendatahub/maas-controller-7496565bcf-9jt85"
Oct 30 19:21:42.939492 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.939461    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a43508c1-d783-41eb-8c54-6dd31e1fafab-webhook-cert\") pod \"maas-controller-7496565bcf-9jt85\" (UID: \"a43508c1-d783-41eb-8c54-6dd31e1fafab\") " pod="opendatahub/maas-controller-7496565bcf-9jt85"
Oct 30 19:21:42.940077 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.940053    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a43508c1-d783-41eb-8c54-6dd31e1fafab-metrics-certs\") pod \"maas-controller-7496565bcf-9jt85\" (UID: \"a43508c1-d783-41eb-8c54-6dd31e1fafab\") " pod="opendatahub/maas-controller-7496565bcf-9jt85"
Oct 30 19:21:42.973567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.973542    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 19:21:42.975071 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.975042    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 19:21:42.996194 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:42.996166    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgqqt\" (UniqueName: \"kubernetes.io/projected/a43508c1-d783-41eb-8c54-6dd31e1fafab-kube-api-access-rgqqt\") pod \"maas-controller-7496565bcf-9jt85\" (UID: \"a43508c1-d783-41eb-8c54-6dd31e1fafab\") " pod="opendatahub/maas-controller-7496565bcf-9jt85"
Oct 30 19:21:43.101065 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:43.101007    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7496565bcf-9jt85"
Oct 30 19:21:43.654075 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:43.654037    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7496565bcf-9jt85"]
Oct 30 19:21:43.666505 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:21:43.656834    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda43508c1_d783_41eb_8c54_6dd31e1fafab.slice/crio-fe43287807f747911c7502619dd36d0ae5d6c3bd11d126d41757340b93878fdf WatchSource:0}: Error finding container fe43287807f747911c7502619dd36d0ae5d6c3bd11d126d41757340b93878fdf: Status 404 returned error can't find the container with id fe43287807f747911c7502619dd36d0ae5d6c3bd11d126d41757340b93878fdf
Oct 30 19:21:43.678627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:43.675181    4544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:21:43.737487 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:43.737459    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7496565bcf-9jt85" event={"ID":"a43508c1-d783-41eb-8c54-6dd31e1fafab","Type":"ContainerStarted","Data":"fe43287807f747911c7502619dd36d0ae5d6c3bd11d126d41757340b93878fdf"}
Oct 30 19:21:45.747930 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:45.747893    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7496565bcf-9jt85" event={"ID":"a43508c1-d783-41eb-8c54-6dd31e1fafab","Type":"ContainerStarted","Data":"7cf41c7af014d9d382b3e3db311a7515b6cbf29bc93512b4bb6bc7b6e5026fbd"}
Oct 30 19:21:45.748335 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:45.747958    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7496565bcf-9jt85"
Oct 30 19:21:45.857166 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:45.857104    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-7496565bcf-9jt85" podStartSLOduration=2.837191522 podStartE2EDuration="3.857085563s" totalImagesPullingTime="1.019894041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:45.856175279 +0000 UTC m=+2703.382050387" watchObservedRunningTime="2026-10-30 19:21:45.857085563 +0000 UTC m=+2703.382960659"
Oct 30 19:21:56.754438 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:21:56.754409    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7496565bcf-9jt85"
Oct 30 19:22:05.532780 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:22:05.532738    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:22:19.679340 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:19.679303    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7496565bcf-9jt85"]
Oct 30 19:22:19.679975 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:19.679562    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-7496565bcf-9jt85" podUID="a43508c1-d783-41eb-8c54-6dd31e1fafab" containerName="manager" containerID="cri-o://7cf41c7af014d9d382b3e3db311a7515b6cbf29bc93512b4bb6bc7b6e5026fbd" gracePeriod=30
Oct 30 19:22:19.908720 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:19.908693    4544 generic.go:356] "Generic (PLEG): container finished" podID="a43508c1-d783-41eb-8c54-6dd31e1fafab" containerID="7cf41c7af014d9d382b3e3db311a7515b6cbf29bc93512b4bb6bc7b6e5026fbd" exitCode=0
Oct 30 19:22:19.908818 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:19.908770    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7496565bcf-9jt85" event={"ID":"a43508c1-d783-41eb-8c54-6dd31e1fafab","Type":"ContainerDied","Data":"7cf41c7af014d9d382b3e3db311a7515b6cbf29bc93512b4bb6bc7b6e5026fbd"}
Oct 30 19:22:19.927879 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:19.927855    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7496565bcf-9jt85"
Oct 30 19:22:20.039701 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:20.039665    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a43508c1-d783-41eb-8c54-6dd31e1fafab-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a43508c1-d783-41eb-8c54-6dd31e1fafab-metrics-certs\") pod \"a43508c1-d783-41eb-8c54-6dd31e1fafab\" (UID: \"a43508c1-d783-41eb-8c54-6dd31e1fafab\") "
Oct 30 19:22:20.039701 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:20.039705    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a43508c1-d783-41eb-8c54-6dd31e1fafab-webhook-cert\" (UniqueName: \"kubernetes.io/secret/a43508c1-d783-41eb-8c54-6dd31e1fafab-webhook-cert\") pod \"a43508c1-d783-41eb-8c54-6dd31e1fafab\" (UID: \"a43508c1-d783-41eb-8c54-6dd31e1fafab\") "
Oct 30 19:22:20.040028 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:20.039779    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a43508c1-d783-41eb-8c54-6dd31e1fafab-kube-api-access-rgqqt\" (UniqueName: \"kubernetes.io/projected/a43508c1-d783-41eb-8c54-6dd31e1fafab-kube-api-access-rgqqt\") pod \"a43508c1-d783-41eb-8c54-6dd31e1fafab\" (UID: \"a43508c1-d783-41eb-8c54-6dd31e1fafab\") "
Oct 30 19:22:20.048840 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:20.048775    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a43508c1-d783-41eb-8c54-6dd31e1fafab-kube-api-access-rgqqt" pod "a43508c1-d783-41eb-8c54-6dd31e1fafab" (UID: "a43508c1-d783-41eb-8c54-6dd31e1fafab"). InnerVolumeSpecName "kube-api-access-rgqqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:22:20.048840 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:20.048780    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a43508c1-d783-41eb-8c54-6dd31e1fafab-webhook-cert" pod "a43508c1-d783-41eb-8c54-6dd31e1fafab" (UID: "a43508c1-d783-41eb-8c54-6dd31e1fafab"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:22:20.049173 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:20.049143    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a43508c1-d783-41eb-8c54-6dd31e1fafab-metrics-certs" pod "a43508c1-d783-41eb-8c54-6dd31e1fafab" (UID: "a43508c1-d783-41eb-8c54-6dd31e1fafab"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:22:20.140812 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:20.140775    4544 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a43508c1-d783-41eb-8c54-6dd31e1fafab-metrics-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:22:20.140812 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:20.140805    4544 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a43508c1-d783-41eb-8c54-6dd31e1fafab-webhook-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:22:20.140812 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:20.140815    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgqqt\" (UniqueName: \"kubernetes.io/projected/a43508c1-d783-41eb-8c54-6dd31e1fafab-kube-api-access-rgqqt\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:22:20.914903 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:20.914824    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7496565bcf-9jt85"
Oct 30 19:22:20.915327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:20.914820    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7496565bcf-9jt85" event={"ID":"a43508c1-d783-41eb-8c54-6dd31e1fafab","Type":"ContainerDied","Data":"fe43287807f747911c7502619dd36d0ae5d6c3bd11d126d41757340b93878fdf"}
Oct 30 19:22:20.915327 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:20.914954    4544 scope.go:122] "RemoveContainer" containerID="7cf41c7af014d9d382b3e3db311a7515b6cbf29bc93512b4bb6bc7b6e5026fbd"
Oct 30 19:22:21.069537 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:21.069502    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7496565bcf-9jt85"]
Oct 30 19:22:21.116074 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:21.116039    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7496565bcf-9jt85"]
Oct 30 19:22:22.472462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.472429    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7496565bcf-r9dth"]
Oct 30 19:22:22.472864 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.472802    4544 cpu_manager.go:414] "containerMap: removing container" podUID="a43508c1-d783-41eb-8c54-6dd31e1fafab" containerName="manager"
Oct 30 19:22:22.472864 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.472814    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a43508c1-d783-41eb-8c54-6dd31e1fafab" containerName="manager"
Oct 30 19:22:22.472942 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.472876    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="a43508c1-d783-41eb-8c54-6dd31e1fafab" containerName="manager"
Oct 30 19:22:22.477101 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.477085    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7496565bcf-r9dth"
Oct 30 19:22:22.565414 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.565379    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb9ff216-796a-44c4-95ee-ce96085894f1-webhook-cert\") pod \"maas-controller-7496565bcf-r9dth\" (UID: \"fb9ff216-796a-44c4-95ee-ce96085894f1\") " pod="opendatahub/maas-controller-7496565bcf-r9dth"
Oct 30 19:22:22.565682 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.565420    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb9ff216-796a-44c4-95ee-ce96085894f1-metrics-certs\") pod \"maas-controller-7496565bcf-r9dth\" (UID: \"fb9ff216-796a-44c4-95ee-ce96085894f1\") " pod="opendatahub/maas-controller-7496565bcf-r9dth"
Oct 30 19:22:22.565682 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.565516    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8z92\" (UniqueName: \"kubernetes.io/projected/fb9ff216-796a-44c4-95ee-ce96085894f1-kube-api-access-z8z92\") pod \"maas-controller-7496565bcf-r9dth\" (UID: \"fb9ff216-796a-44c4-95ee-ce96085894f1\") " pod="opendatahub/maas-controller-7496565bcf-r9dth"
Oct 30 19:22:22.572804 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.572768    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 19:22:22.572923 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.572839    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vkhgq\""
Oct 30 19:22:22.572923 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.572844    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 19:22:22.572923 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.572877    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 19:22:22.574660 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.574636    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7496565bcf-r9dth"]
Oct 30 19:22:22.601250 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.601222    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a43508c1-d783-41eb-8c54-6dd31e1fafab" path="/var/lib/kubelet/pods/a43508c1-d783-41eb-8c54-6dd31e1fafab/volumes"
Oct 30 19:22:22.667072 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.667035    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb9ff216-796a-44c4-95ee-ce96085894f1-webhook-cert\") pod \"maas-controller-7496565bcf-r9dth\" (UID: \"fb9ff216-796a-44c4-95ee-ce96085894f1\") " pod="opendatahub/maas-controller-7496565bcf-r9dth"
Oct 30 19:22:22.667233 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.667095    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb9ff216-796a-44c4-95ee-ce96085894f1-metrics-certs\") pod \"maas-controller-7496565bcf-r9dth\" (UID: \"fb9ff216-796a-44c4-95ee-ce96085894f1\") " pod="opendatahub/maas-controller-7496565bcf-r9dth"
Oct 30 19:22:22.667233 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.667158    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8z92\" (UniqueName: \"kubernetes.io/projected/fb9ff216-796a-44c4-95ee-ce96085894f1-kube-api-access-z8z92\") pod \"maas-controller-7496565bcf-r9dth\" (UID: \"fb9ff216-796a-44c4-95ee-ce96085894f1\") " pod="opendatahub/maas-controller-7496565bcf-r9dth"
Oct 30 19:22:22.669499 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.669475    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb9ff216-796a-44c4-95ee-ce96085894f1-webhook-cert\") pod \"maas-controller-7496565bcf-r9dth\" (UID: \"fb9ff216-796a-44c4-95ee-ce96085894f1\") " pod="opendatahub/maas-controller-7496565bcf-r9dth"
Oct 30 19:22:22.669627 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.669516    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb9ff216-796a-44c4-95ee-ce96085894f1-metrics-certs\") pod \"maas-controller-7496565bcf-r9dth\" (UID: \"fb9ff216-796a-44c4-95ee-ce96085894f1\") " pod="opendatahub/maas-controller-7496565bcf-r9dth"
Oct 30 19:22:22.722095 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.722061    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8z92\" (UniqueName: \"kubernetes.io/projected/fb9ff216-796a-44c4-95ee-ce96085894f1-kube-api-access-z8z92\") pod \"maas-controller-7496565bcf-r9dth\" (UID: \"fb9ff216-796a-44c4-95ee-ce96085894f1\") " pod="opendatahub/maas-controller-7496565bcf-r9dth"
Oct 30 19:22:22.790308 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:22.790279    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7496565bcf-r9dth"
Oct 30 19:22:23.155514 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:23.155410    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7496565bcf-r9dth"]
Oct 30 19:22:23.186166 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:22:23.186139    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb9ff216_796a_44c4_95ee_ce96085894f1.slice/crio-20bae3753a5fe6609705b699f51d9e1ef9b360bbc33dc4bc072e4ac3c22cab13 WatchSource:0}: Error finding container 20bae3753a5fe6609705b699f51d9e1ef9b360bbc33dc4bc072e4ac3c22cab13: Status 404 returned error can't find the container with id 20bae3753a5fe6609705b699f51d9e1ef9b360bbc33dc4bc072e4ac3c22cab13
Oct 30 19:22:23.930065 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:23.930029    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7496565bcf-r9dth" event={"ID":"fb9ff216-796a-44c4-95ee-ce96085894f1","Type":"ContainerStarted","Data":"20bae3753a5fe6609705b699f51d9e1ef9b360bbc33dc4bc072e4ac3c22cab13"}
Oct 30 19:22:24.935838 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:24.935800    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7496565bcf-r9dth" event={"ID":"fb9ff216-796a-44c4-95ee-ce96085894f1","Type":"ContainerStarted","Data":"b27c3dfd8fce84c977e337e5225e44ce83fb828bac01a248dcc136705f96e10f"}
Oct 30 19:22:24.936323 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:24.935879    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7496565bcf-r9dth"
Oct 30 19:22:25.055996 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:25.055936    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-7496565bcf-r9dth" podStartSLOduration=2.183389405 podStartE2EDuration="3.055921717s" totalImagesPullingTime="872.532312ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:25.054317785 +0000 UTC m=+2742.580192883" watchObservedRunningTime="2026-10-30 19:22:25.055921717 +0000 UTC m=+2742.581796848"
Oct 30 19:22:35.531176 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:22:35.531131    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:22:35.943202 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:22:35.943133    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7496565bcf-r9dth"
Oct 30 19:23:05.530544 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:23:05.530500    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:23:35.532510 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:23:35.532471    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:24:05.531870 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:24:05.531781    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:24:35.533396 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:24:35.533356    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:25:05.531233 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:25:05.531187    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:25:35.531900 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:25:35.531855    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:26:05.531003 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:26:05.530967    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:26:35.530409 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:26:35.530371    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:26:43.007965 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:26:43.007931    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 19:26:43.012214 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:26:43.012193    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 19:27:05.531129 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:27:05.531084    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:27:35.530336 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:27:35.530297    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:28:05.531124 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:28:05.531080    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:28:11.871714 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:11.871676    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"]
Oct 30 19:28:11.875503 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:11.875480    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:11.973870 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:11.973838    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-7d30a0-openshift-default-dockercfg-rx5nq\""
Oct 30 19:28:12.023299 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.023194    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"]
Oct 30 19:28:12.026965 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.026938    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-data\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.027128 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.026996    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-credential-socket\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.027128 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.027059    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baaa260b-f29d-4e58-b900-d8bd50c382d0-istiod-ca-cert\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.027240 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.027135    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-workload-certs\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.027240 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.027163    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5fhr\" (UniqueName: \"kubernetes.io/projected/baaa260b-f29d-4e58-b900-d8bd50c382d0-kube-api-access-w5fhr\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.027240 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.027186    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-token\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.027240 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.027204    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-envoy\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.027413 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.027244    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-workload-socket\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.027461 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.027410    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-podinfo\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.128960 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.128876    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-workload-socket\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.128960 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.128958    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-podinfo\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.129201 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.128987    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-data\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.129201 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.129018    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-credential-socket\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.129201 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.129047    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baaa260b-f29d-4e58-b900-d8bd50c382d0-istiod-ca-cert\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.129201 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.129083    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-workload-certs\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.129201 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.129106    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5fhr\" (UniqueName: \"kubernetes.io/projected/baaa260b-f29d-4e58-b900-d8bd50c382d0-kube-api-access-w5fhr\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.129201 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.129150    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-token\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.129201 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.129185    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-envoy\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.129544 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.129317    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-workload-socket\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.129544 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.129406    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-data\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.129642 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.129570    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-workload-certs\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.129706 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.129677    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-credential-socket\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.130448 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.130424    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baaa260b-f29d-4e58-b900-d8bd50c382d0-istiod-ca-cert\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.131439 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.131415    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-podinfo\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.131541 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.131510    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-envoy\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.195260 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.195233    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-token\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.195795 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.195774    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5fhr\" (UniqueName: \"kubernetes.io/projected/baaa260b-f29d-4e58-b900-d8bd50c382d0-kube-api-access-w5fhr\") pod \"e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") " pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.491314 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.491243    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:12.908554 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.908522    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"]
Oct 30 19:28:12.943816 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:28:12.943790    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaaa260b_f29d_4e58_b900_d8bd50c382d0.slice/crio-020196a2af9f4098fa9b9944eb55778ab7a21778c14c9db3cfc44c64458c377a WatchSource:0}: Error finding container 020196a2af9f4098fa9b9944eb55778ab7a21778c14c9db3cfc44c64458c377a: Status 404 returned error can't find the container with id 020196a2af9f4098fa9b9944eb55778ab7a21778c14c9db3cfc44c64458c377a
Oct 30 19:28:12.946012 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.945954    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:28:12.946106 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.946049    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:28:12.946106 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:12.946075    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:28:13.650080 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:13.650045    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5" event={"ID":"baaa260b-f29d-4e58-b900-d8bd50c382d0","Type":"ContainerStarted","Data":"bc84a90791dda76e4ce15bf72dd426a6724eb6c4f5d4b309664032c343384317"}
Oct 30 19:28:13.650267 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:13.650094    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5" event={"ID":"baaa260b-f29d-4e58-b900-d8bd50c382d0","Type":"ContainerStarted","Data":"020196a2af9f4098fa9b9944eb55778ab7a21778c14c9db3cfc44c64458c377a"}
Oct 30 19:28:13.853070 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:13.853009    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5" podStartSLOduration=2.852993401 podStartE2EDuration="2.852993401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:13.852189305 +0000 UTC m=+3091.378064401" watchObservedRunningTime="2026-10-30 19:28:13.852993401 +0000 UTC m=+3091.378868497"
Oct 30 19:28:14.492413 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:14.492383    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:14.497423 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:14.497397    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:14.653957 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:14.653927    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:14.655065 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:14.655035    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:20.288782 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:20.288750    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg"]
Oct 30 19:28:20.292205 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:20.292188    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg"
Oct 30 19:28:20.390990 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:20.390958    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-7d30a0\""
Oct 30 19:28:20.390990 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:20.390958    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-721d9a0c-dockercfg-b9qll\""
Oct 30 19:28:20.399591 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:20.393558    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg"]
Oct 30 19:28:20.410968 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:20.410937    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/851f1425-8156-4269-9162-9ed1d56502e6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg\" (UID: \"851f1425-8156-4269-9162-9ed1d56502e6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg"
Oct 30 19:28:20.411107 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:20.410973    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwntn\" (UniqueName: \"kubernetes.io/projected/851f1425-8156-4269-9162-9ed1d56502e6-kube-api-access-cwntn\") pod \"payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg\" (UID: \"851f1425-8156-4269-9162-9ed1d56502e6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg"
Oct 30 19:28:20.512236 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:20.512190    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/851f1425-8156-4269-9162-9ed1d56502e6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg\" (UID: \"851f1425-8156-4269-9162-9ed1d56502e6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg"
Oct 30 19:28:20.512393 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:20.512257    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwntn\" (UniqueName: \"kubernetes.io/projected/851f1425-8156-4269-9162-9ed1d56502e6-kube-api-access-cwntn\") pod \"payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg\" (UID: \"851f1425-8156-4269-9162-9ed1d56502e6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg"
Oct 30 19:28:20.512940 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:20.512916    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/851f1425-8156-4269-9162-9ed1d56502e6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg\" (UID: \"851f1425-8156-4269-9162-9ed1d56502e6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg"
Oct 30 19:28:20.565567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:20.565503    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwntn\" (UniqueName: \"kubernetes.io/projected/851f1425-8156-4269-9162-9ed1d56502e6-kube-api-access-cwntn\") pod \"payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg\" (UID: \"851f1425-8156-4269-9162-9ed1d56502e6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg"
Oct 30 19:28:20.606004 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:20.605981    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg"
Oct 30 19:28:20.972235 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:20.972158    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg"]
Oct 30 19:28:20.975560 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:28:20.975525    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod851f1425_8156_4269_9162_9ed1d56502e6.slice/crio-14f8d5aab9979b0abab53e3fcca30300aa50d13e156bf45fd47268a1f6b8a4f4 WatchSource:0}: Error finding container 14f8d5aab9979b0abab53e3fcca30300aa50d13e156bf45fd47268a1f6b8a4f4: Status 404 returned error can't find the container with id 14f8d5aab9979b0abab53e3fcca30300aa50d13e156bf45fd47268a1f6b8a4f4
Oct 30 19:28:21.686869 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:21.686834    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg" event={"ID":"851f1425-8156-4269-9162-9ed1d56502e6","Type":"ContainerStarted","Data":"d79a6ed8c1722c23da5c1d85551a8da239f092ab4467b663ced62c7f662679ef"}
Oct 30 19:28:21.687261 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:21.686884    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg" event={"ID":"851f1425-8156-4269-9162-9ed1d56502e6","Type":"ContainerStarted","Data":"14f8d5aab9979b0abab53e3fcca30300aa50d13e156bf45fd47268a1f6b8a4f4"}
Oct 30 19:28:21.687261 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:21.686914    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg"
Oct 30 19:28:21.793922 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:21.793874    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg" podStartSLOduration=1.793859958 podStartE2EDuration="1.793859958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:21.792545916 +0000 UTC m=+3099.318421016" watchObservedRunningTime="2026-10-30 19:28:21.793859958 +0000 UTC m=+3099.319735054"
Oct 30 19:28:23.066874 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:23.066796    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"]
Oct 30 19:28:23.070362 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:23.070345    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:28:23.119169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:23.119139    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-7d30a0-serving-cert\""
Oct 30 19:28:23.137461 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:23.137437    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqhg4\" (UniqueName: \"kubernetes.io/projected/47e6daf7-cd4f-4688-a878-194fc432f2fc-kube-api-access-pqhg4\") pod \"maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf\" (UID: \"47e6daf7-cd4f-4688-a878-194fc432f2fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:28:23.137583 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:23.137475    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47e6daf7-cd4f-4688-a878-194fc432f2fc-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf\" (UID: \"47e6daf7-cd4f-4688-a878-194fc432f2fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:28:23.137583 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:23.137550    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47e6daf7-cd4f-4688-a878-194fc432f2fc-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf\" (UID: \"47e6daf7-cd4f-4688-a878-194fc432f2fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:28:23.137750 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:23.137682    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47e6daf7-cd4f-4688-a878-194fc432f2fc-metrics-certs\") pod \"maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf\" (UID: \"47e6daf7-cd4f-4688-a878-194fc432f2fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:28:23.167060 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:23.167023    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"]
Oct 30 19:28:23.238218 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:23.238187    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqhg4\" (UniqueName: \"kubernetes.io/projected/47e6daf7-cd4f-4688-a878-194fc432f2fc-kube-api-access-pqhg4\") pod \"maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf\" (UID: \"47e6daf7-cd4f-4688-a878-194fc432f2fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:28:23.238382 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:23.238231    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47e6daf7-cd4f-4688-a878-194fc432f2fc-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf\" (UID: \"47e6daf7-cd4f-4688-a878-194fc432f2fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:28:23.238382 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:23.238283    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47e6daf7-cd4f-4688-a878-194fc432f2fc-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf\" (UID: \"47e6daf7-cd4f-4688-a878-194fc432f2fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:28:23.238382 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:23.238323    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47e6daf7-cd4f-4688-a878-194fc432f2fc-metrics-certs\") pod \"maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf\" (UID: \"47e6daf7-cd4f-4688-a878-194fc432f2fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:28:23.238511 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:28:23.238413    4544 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-serving-cert: secret "maas-api-e2e-worker-single-main-7d30a0-serving-cert" not found
Oct 30 19:28:23.238511 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:28:23.238473    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47e6daf7-cd4f-4688-a878-194fc432f2fc-maas-api-tls podName:47e6daf7-cd4f-4688-a878-194fc432f2fc nodeName:}" failed. No retries permitted until 2026-10-30 19:28:23.738455696 +0000 UTC m=+3101.264330770 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/47e6daf7-cd4f-4688-a878-194fc432f2fc-maas-api-tls") pod "maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf" (UID: "47e6daf7-cd4f-4688-a878-194fc432f2fc") : secret "maas-api-e2e-worker-single-main-7d30a0-serving-cert" not found
Oct 30 19:28:23.239044 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:23.239016    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47e6daf7-cd4f-4688-a878-194fc432f2fc-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf\" (UID: \"47e6daf7-cd4f-4688-a878-194fc432f2fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:28:23.240930 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:23.240904    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47e6daf7-cd4f-4688-a878-194fc432f2fc-metrics-certs\") pod \"maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf\" (UID: \"47e6daf7-cd4f-4688-a878-194fc432f2fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:28:23.291837 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:23.291815    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqhg4\" (UniqueName: \"kubernetes.io/projected/47e6daf7-cd4f-4688-a878-194fc432f2fc-kube-api-access-pqhg4\") pod \"maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf\" (UID: \"47e6daf7-cd4f-4688-a878-194fc432f2fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:28:23.743423 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:23.743390    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47e6daf7-cd4f-4688-a878-194fc432f2fc-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf\" (UID: \"47e6daf7-cd4f-4688-a878-194fc432f2fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:28:23.745752 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:23.745727    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47e6daf7-cd4f-4688-a878-194fc432f2fc-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf\" (UID: \"47e6daf7-cd4f-4688-a878-194fc432f2fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:28:23.983117 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:23.983084    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:28:24.351209 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:24.351174    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"]
Oct 30 19:28:24.354030 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:28:24.353992    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47e6daf7_cd4f_4688_a878_194fc432f2fc.slice/crio-fd0510575d431a6d95c0ca294b30ffeb8e0bee55f5a1366bf873b29b7b37404f WatchSource:0}: Error finding container fd0510575d431a6d95c0ca294b30ffeb8e0bee55f5a1366bf873b29b7b37404f: Status 404 returned error can't find the container with id fd0510575d431a6d95c0ca294b30ffeb8e0bee55f5a1366bf873b29b7b37404f
Oct 30 19:28:24.355597 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:24.355574    4544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:28:24.700594 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:24.700519    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf" event={"ID":"47e6daf7-cd4f-4688-a878-194fc432f2fc","Type":"ContainerStarted","Data":"fd0510575d431a6d95c0ca294b30ffeb8e0bee55f5a1366bf873b29b7b37404f"}
Oct 30 19:28:25.706530 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:25.706490    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf" event={"ID":"47e6daf7-cd4f-4688-a878-194fc432f2fc","Type":"ContainerStarted","Data":"109d7502c4e637947c45dcbb7e7231da895d059f338e68631b8abe0f919a8c5c"}
Oct 30 19:28:25.706947 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:25.706638    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:28:25.862542 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:25.862490    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf" podStartSLOduration=1.98035255 podStartE2EDuration="2.862476996s" totalImagesPullingTime="882.124446ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:25.815103278 +0000 UTC m=+3103.340978369" watchObservedRunningTime="2026-10-30 19:28:25.862476996 +0000 UTC m=+3103.388352092"
Oct 30 19:28:31.717492 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:31.717461    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:28:32.694402 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:32.694372    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg"
Oct 30 19:28:35.530839 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:28:35.530799    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:28:40.781132 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:40.781092    4544 generic.go:356] "Generic (PLEG): container finished" podID="baaa260b-f29d-4e58-b900-d8bd50c382d0" containerID="bc84a90791dda76e4ce15bf72dd426a6724eb6c4f5d4b309664032c343384317" exitCode=0
Oct 30 19:28:40.781573 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:40.781142    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5" event={"ID":"baaa260b-f29d-4e58-b900-d8bd50c382d0","Type":"ContainerDied","Data":"bc84a90791dda76e4ce15bf72dd426a6724eb6c4f5d4b309664032c343384317"}
Oct 30 19:28:40.781573 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:40.781486    4544 scope.go:122] "RemoveContainer" containerID="bc84a90791dda76e4ce15bf72dd426a6724eb6c4f5d4b309664032c343384317"
Oct 30 19:28:40.782660 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:40.782633    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:28:40.782722 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:40.782698    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:28:40.782758 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:40.782728    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:28:41.787449 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:41.787411    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5" event={"ID":"baaa260b-f29d-4e58-b900-d8bd50c382d0","Type":"ContainerStarted","Data":"d61db4d888308bac216a856fab4da95aa14f328d2e193dbaa64563f12f0d6de1"}
Oct 30 19:28:42.491770 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:42.491732    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:42.493133 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:42.493107    4544 patch_prober.go:29] interesting pod/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 19:28:42.493216 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:42.493156    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5" podUID="baaa260b-f29d-4e58-b900-d8bd50c382d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 19:28:43.492468 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:43.492432    4544 patch_prober.go:29] interesting pod/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 19:28:43.493011 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:43.492484    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5" podUID="baaa260b-f29d-4e58-b900-d8bd50c382d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 19:28:44.491901 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:44.491861    4544 patch_prober.go:29] interesting pod/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 19:28:44.492089 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:44.491917    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5" podUID="baaa260b-f29d-4e58-b900-d8bd50c382d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 19:28:44.654226 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:44.654192    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:45.491945 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:45.491908    4544 patch_prober.go:29] interesting pod/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 19:28:45.492134 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:45.491963    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5" podUID="baaa260b-f29d-4e58-b900-d8bd50c382d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 19:28:46.492034 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:46.491994    4544 patch_prober.go:29] interesting pod/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 30 19:28:46.492521 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:46.492058    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5" podUID="baaa260b-f29d-4e58-b900-d8bd50c382d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 30 19:28:47.496247 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:47.496218    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:28:47.497239 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:28:47.497213    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:29:05.530236 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:29:05.530177    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:29:35.530895 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:29:35.530853    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:29:45.208591 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:45.208559    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg"]
Oct 30 19:29:50.209253 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:50.209187    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg" podUID="851f1425-8156-4269-9162-9ed1d56502e6" containerName="payload-pre-processing" containerID="cri-o://d79a6ed8c1722c23da5c1d85551a8da239f092ab4467b663ced62c7f662679ef" gracePeriod=25
Oct 30 19:29:51.371624 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:51.371569    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"]
Oct 30 19:29:51.372088 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:51.371767    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf" podUID="47e6daf7-cd4f-4688-a878-194fc432f2fc" containerName="maas-api" containerID="cri-o://109d7502c4e637947c45dcbb7e7231da895d059f338e68631b8abe0f919a8c5c" gracePeriod=30
Oct 30 19:29:51.712021 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:51.711934    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf" podUID="47e6daf7-cd4f-4688-a878-194fc432f2fc" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.131:8443/health\": dial tcp 10.133.0.131:8443: connect: connection refused"
Oct 30 19:29:52.631721 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:52.631696    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:29:52.692636 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:52.692576    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg" podUID="851f1425-8156-4269-9162-9ed1d56502e6" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:29:52.732833 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:52.732758    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/47e6daf7-cd4f-4688-a878-194fc432f2fc-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47e6daf7-cd4f-4688-a878-194fc432f2fc-openshift-service-ca\") pod \"47e6daf7-cd4f-4688-a878-194fc432f2fc\" (UID: \"47e6daf7-cd4f-4688-a878-194fc432f2fc\") "
Oct 30 19:29:52.732976 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:52.732880    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/47e6daf7-cd4f-4688-a878-194fc432f2fc-metrics-certs\" (UniqueName: \"kubernetes.io/secret/47e6daf7-cd4f-4688-a878-194fc432f2fc-metrics-certs\") pod \"47e6daf7-cd4f-4688-a878-194fc432f2fc\" (UID: \"47e6daf7-cd4f-4688-a878-194fc432f2fc\") "
Oct 30 19:29:52.732976 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:52.732904    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47e6daf7-cd4f-4688-a878-194fc432f2fc-kube-api-access-pqhg4\" (UniqueName: \"kubernetes.io/projected/47e6daf7-cd4f-4688-a878-194fc432f2fc-kube-api-access-pqhg4\") pod \"47e6daf7-cd4f-4688-a878-194fc432f2fc\" (UID: \"47e6daf7-cd4f-4688-a878-194fc432f2fc\") "
Oct 30 19:29:52.732976 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:52.732921    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/47e6daf7-cd4f-4688-a878-194fc432f2fc-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47e6daf7-cd4f-4688-a878-194fc432f2fc-maas-api-tls\") pod \"47e6daf7-cd4f-4688-a878-194fc432f2fc\" (UID: \"47e6daf7-cd4f-4688-a878-194fc432f2fc\") "
Oct 30 19:29:52.733133 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:52.733102    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47e6daf7-cd4f-4688-a878-194fc432f2fc-openshift-service-ca" pod "47e6daf7-cd4f-4688-a878-194fc432f2fc" (UID: "47e6daf7-cd4f-4688-a878-194fc432f2fc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:29:52.733271 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:52.733250    4544 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47e6daf7-cd4f-4688-a878-194fc432f2fc-openshift-service-ca\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:52.735057 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:52.735025    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47e6daf7-cd4f-4688-a878-194fc432f2fc-maas-api-tls" pod "47e6daf7-cd4f-4688-a878-194fc432f2fc" (UID: "47e6daf7-cd4f-4688-a878-194fc432f2fc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:29:52.735155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:52.735094    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47e6daf7-cd4f-4688-a878-194fc432f2fc-kube-api-access-pqhg4" pod "47e6daf7-cd4f-4688-a878-194fc432f2fc" (UID: "47e6daf7-cd4f-4688-a878-194fc432f2fc"). InnerVolumeSpecName "kube-api-access-pqhg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:29:52.735155 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:52.735125    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47e6daf7-cd4f-4688-a878-194fc432f2fc-metrics-certs" pod "47e6daf7-cd4f-4688-a878-194fc432f2fc" (UID: "47e6daf7-cd4f-4688-a878-194fc432f2fc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:29:52.834173 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:52.834136    4544 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/47e6daf7-cd4f-4688-a878-194fc432f2fc-metrics-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:52.834173 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:52.834169    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqhg4\" (UniqueName: \"kubernetes.io/projected/47e6daf7-cd4f-4688-a878-194fc432f2fc-kube-api-access-pqhg4\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:52.834173 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:52.834180    4544 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47e6daf7-cd4f-4688-a878-194fc432f2fc-maas-api-tls\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:53.117314 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:53.117272    4544 generic.go:356] "Generic (PLEG): container finished" podID="47e6daf7-cd4f-4688-a878-194fc432f2fc" containerID="109d7502c4e637947c45dcbb7e7231da895d059f338e68631b8abe0f919a8c5c" exitCode=0
Oct 30 19:29:53.117564 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:53.117339    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"
Oct 30 19:29:53.117564 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:53.117359    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf" event={"ID":"47e6daf7-cd4f-4688-a878-194fc432f2fc","Type":"ContainerDied","Data":"109d7502c4e637947c45dcbb7e7231da895d059f338e68631b8abe0f919a8c5c"}
Oct 30 19:29:53.117564 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:53.117496    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf" event={"ID":"47e6daf7-cd4f-4688-a878-194fc432f2fc","Type":"ContainerDied","Data":"fd0510575d431a6d95c0ca294b30ffeb8e0bee55f5a1366bf873b29b7b37404f"}
Oct 30 19:29:53.117564 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:53.117531    4544 scope.go:122] "RemoveContainer" containerID="109d7502c4e637947c45dcbb7e7231da895d059f338e68631b8abe0f919a8c5c"
Oct 30 19:29:53.127860 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:53.127841    4544 scope.go:122] "RemoveContainer" containerID="109d7502c4e637947c45dcbb7e7231da895d059f338e68631b8abe0f919a8c5c"
Oct 30 19:29:53.128151 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:29:53.128131    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"109d7502c4e637947c45dcbb7e7231da895d059f338e68631b8abe0f919a8c5c\": container with ID starting with 109d7502c4e637947c45dcbb7e7231da895d059f338e68631b8abe0f919a8c5c not found: ID does not exist" containerID="109d7502c4e637947c45dcbb7e7231da895d059f338e68631b8abe0f919a8c5c"
Oct 30 19:29:53.128220 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:53.128162    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"109d7502c4e637947c45dcbb7e7231da895d059f338e68631b8abe0f919a8c5c"} err="failed to get container status \"109d7502c4e637947c45dcbb7e7231da895d059f338e68631b8abe0f919a8c5c\": rpc error: code = NotFound desc = could not find container \"109d7502c4e637947c45dcbb7e7231da895d059f338e68631b8abe0f919a8c5c\": container with ID starting with 109d7502c4e637947c45dcbb7e7231da895d059f338e68631b8abe0f919a8c5c not found: ID does not exist"
Oct 30 19:29:53.271561 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:53.271523    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"]
Oct 30 19:29:53.317787 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:53.317754    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7d30a0-555567bf88-w8sjf"]
Oct 30 19:29:54.601158 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:29:54.601124    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="47e6daf7-cd4f-4688-a878-194fc432f2fc" path="/var/lib/kubelet/pods/47e6daf7-cd4f-4688-a878-194fc432f2fc/volumes"
Oct 30 19:30:00.999066 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:00.999029    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"]
Oct 30 19:30:00.999542 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:00.999245    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5" podUID="baaa260b-f29d-4e58-b900-d8bd50c382d0" containerName="istio-proxy" containerID="cri-o://d61db4d888308bac216a856fab4da95aa14f328d2e193dbaa64563f12f0d6de1" gracePeriod=30
Oct 30 19:30:02.498982 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:02.498938    4544 patch_prober.go:29] interesting pod/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:30:02.499367 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:02.499007    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5" podUID="baaa260b-f29d-4e58-b900-d8bd50c382d0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:30:02.693336 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:02.693291    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg" podUID="851f1425-8156-4269-9162-9ed1d56502e6" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:30:05.531550 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:30:05.531502    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:30:06.183567 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.183538    4544 generic.go:356] "Generic (PLEG): container finished" podID="baaa260b-f29d-4e58-b900-d8bd50c382d0" containerID="d61db4d888308bac216a856fab4da95aa14f328d2e193dbaa64563f12f0d6de1" exitCode=0
Oct 30 19:30:06.183808 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.183611    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5" event={"ID":"baaa260b-f29d-4e58-b900-d8bd50c382d0","Type":"ContainerDied","Data":"d61db4d888308bac216a856fab4da95aa14f328d2e193dbaa64563f12f0d6de1"}
Oct 30 19:30:06.183808 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.183674    4544 scope.go:122] "RemoveContainer" containerID="bc84a90791dda76e4ce15bf72dd426a6724eb6c4f5d4b309664032c343384317"
Oct 30 19:30:06.247353 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.247330    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:30:06.365628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.365588    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-envoy\") pod \"baaa260b-f29d-4e58-b900-d8bd50c382d0\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") "
Oct 30 19:30:06.365812 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.365649    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-credential-socket\") pod \"baaa260b-f29d-4e58-b900-d8bd50c382d0\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") "
Oct 30 19:30:06.365812 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.365695    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-podinfo\") pod \"baaa260b-f29d-4e58-b900-d8bd50c382d0\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") "
Oct 30 19:30:06.365812 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.365712    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-data\") pod \"baaa260b-f29d-4e58-b900-d8bd50c382d0\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") "
Oct 30 19:30:06.365812 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.365755    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/baaa260b-f29d-4e58-b900-d8bd50c382d0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baaa260b-f29d-4e58-b900-d8bd50c382d0-istiod-ca-cert\") pod \"baaa260b-f29d-4e58-b900-d8bd50c382d0\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") "
Oct 30 19:30:06.365812 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.365786    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-workload-socket\") pod \"baaa260b-f29d-4e58-b900-d8bd50c382d0\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") "
Oct 30 19:30:06.366156 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.365832    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/baaa260b-f29d-4e58-b900-d8bd50c382d0-kube-api-access-w5fhr\" (UniqueName: \"kubernetes.io/projected/baaa260b-f29d-4e58-b900-d8bd50c382d0-kube-api-access-w5fhr\") pod \"baaa260b-f29d-4e58-b900-d8bd50c382d0\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") "
Oct 30 19:30:06.366156 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.365869    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-workload-certs\") pod \"baaa260b-f29d-4e58-b900-d8bd50c382d0\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") "
Oct 30 19:30:06.366156 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.365964    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-token\" (UniqueName: \"kubernetes.io/projected/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-token\") pod \"baaa260b-f29d-4e58-b900-d8bd50c382d0\" (UID: \"baaa260b-f29d-4e58-b900-d8bd50c382d0\") "
Oct 30 19:30:06.366156 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.365973    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-credential-socket" pod "baaa260b-f29d-4e58-b900-d8bd50c382d0" (UID: "baaa260b-f29d-4e58-b900-d8bd50c382d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:06.366347 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.366171    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-data" pod "baaa260b-f29d-4e58-b900-d8bd50c382d0" (UID: "baaa260b-f29d-4e58-b900-d8bd50c382d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:06.366347 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.366128    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-workload-socket" pod "baaa260b-f29d-4e58-b900-d8bd50c382d0" (UID: "baaa260b-f29d-4e58-b900-d8bd50c382d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:06.366347 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.366187    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-workload-certs" pod "baaa260b-f29d-4e58-b900-d8bd50c382d0" (UID: "baaa260b-f29d-4e58-b900-d8bd50c382d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:06.366347 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.366295    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:06.366347 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.366319    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:06.366347 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.366340    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:06.366656 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.366353    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:06.366656 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.366536    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baaa260b-f29d-4e58-b900-d8bd50c382d0-istiod-ca-cert" pod "baaa260b-f29d-4e58-b900-d8bd50c382d0" (UID: "baaa260b-f29d-4e58-b900-d8bd50c382d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:06.367996 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.367974    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-podinfo" pod "baaa260b-f29d-4e58-b900-d8bd50c382d0" (UID: "baaa260b-f29d-4e58-b900-d8bd50c382d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:30:06.368172 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.368151    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-envoy" pod "baaa260b-f29d-4e58-b900-d8bd50c382d0" (UID: "baaa260b-f29d-4e58-b900-d8bd50c382d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:06.368234 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.368155    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baaa260b-f29d-4e58-b900-d8bd50c382d0-kube-api-access-w5fhr" pod "baaa260b-f29d-4e58-b900-d8bd50c382d0" (UID: "baaa260b-f29d-4e58-b900-d8bd50c382d0"). InnerVolumeSpecName "kube-api-access-w5fhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:06.368416 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.368396    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-token" pod "baaa260b-f29d-4e58-b900-d8bd50c382d0" (UID: "baaa260b-f29d-4e58-b900-d8bd50c382d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:06.467515 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.467484    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baaa260b-f29d-4e58-b900-d8bd50c382d0-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:06.467515 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.467511    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5fhr\" (UniqueName: \"kubernetes.io/projected/baaa260b-f29d-4e58-b900-d8bd50c382d0-kube-api-access-w5fhr\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:06.467515 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.467523    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:06.467778 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.467538    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:06.467778 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:06.467548    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baaa260b-f29d-4e58-b900-d8bd50c382d0-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:07.189376 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:07.189337    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5" event={"ID":"baaa260b-f29d-4e58-b900-d8bd50c382d0","Type":"ContainerDied","Data":"020196a2af9f4098fa9b9944eb55778ab7a21778c14c9db3cfc44c64458c377a"}
Oct 30 19:30:07.189376 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:07.189358    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"
Oct 30 19:30:07.189892 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:07.189394    4544 scope.go:122] "RemoveContainer" containerID="d61db4d888308bac216a856fab4da95aa14f328d2e193dbaa64563f12f0d6de1"
Oct 30 19:30:07.390572 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:07.390541    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"]
Oct 30 19:30:07.436484 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:07.436454    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-7d30a0-openshift-default-869bc85c7cd7pz5"]
Oct 30 19:30:08.600396 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:08.600358    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="baaa260b-f29d-4e58-b900-d8bd50c382d0" path="/var/lib/kubelet/pods/baaa260b-f29d-4e58-b900-d8bd50c382d0/volumes"
Oct 30 19:30:10.476201 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:10.476136    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg"
Oct 30 19:30:10.609423 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:10.609393    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/851f1425-8156-4269-9162-9ed1d56502e6-kube-api-access-cwntn\" (UniqueName: \"kubernetes.io/projected/851f1425-8156-4269-9162-9ed1d56502e6-kube-api-access-cwntn\") pod \"851f1425-8156-4269-9162-9ed1d56502e6\" (UID: \"851f1425-8156-4269-9162-9ed1d56502e6\") "
Oct 30 19:30:10.609578 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:10.609444    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/851f1425-8156-4269-9162-9ed1d56502e6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/851f1425-8156-4269-9162-9ed1d56502e6-plugins-config-volume\") pod \"851f1425-8156-4269-9162-9ed1d56502e6\" (UID: \"851f1425-8156-4269-9162-9ed1d56502e6\") "
Oct 30 19:30:10.609885 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:10.609865    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/851f1425-8156-4269-9162-9ed1d56502e6-plugins-config-volume" pod "851f1425-8156-4269-9162-9ed1d56502e6" (UID: "851f1425-8156-4269-9162-9ed1d56502e6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:10.611613 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:10.611577    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/851f1425-8156-4269-9162-9ed1d56502e6-kube-api-access-cwntn" pod "851f1425-8156-4269-9162-9ed1d56502e6" (UID: "851f1425-8156-4269-9162-9ed1d56502e6"). InnerVolumeSpecName "kube-api-access-cwntn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:10.711125 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:10.711092    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwntn\" (UniqueName: \"kubernetes.io/projected/851f1425-8156-4269-9162-9ed1d56502e6-kube-api-access-cwntn\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:10.711125 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:10.711124    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/851f1425-8156-4269-9162-9ed1d56502e6-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:11.211238 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:11.211137    4544 generic.go:356] "Generic (PLEG): container finished" podID="851f1425-8156-4269-9162-9ed1d56502e6" containerID="d79a6ed8c1722c23da5c1d85551a8da239f092ab4467b663ced62c7f662679ef" exitCode=0
Oct 30 19:30:11.211238 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:11.211199    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg"
Oct 30 19:30:11.211479 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:11.211265    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg" event={"ID":"851f1425-8156-4269-9162-9ed1d56502e6","Type":"ContainerDied","Data":"d79a6ed8c1722c23da5c1d85551a8da239f092ab4467b663ced62c7f662679ef"}
Oct 30 19:30:11.211479 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:11.211319    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg" event={"ID":"851f1425-8156-4269-9162-9ed1d56502e6","Type":"ContainerDied","Data":"14f8d5aab9979b0abab53e3fcca30300aa50d13e156bf45fd47268a1f6b8a4f4"}
Oct 30 19:30:11.211479 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:11.211349    4544 scope.go:122] "RemoveContainer" containerID="d79a6ed8c1722c23da5c1d85551a8da239f092ab4467b663ced62c7f662679ef"
Oct 30 19:30:11.220072 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:11.220056    4544 scope.go:122] "RemoveContainer" containerID="d79a6ed8c1722c23da5c1d85551a8da239f092ab4467b663ced62c7f662679ef"
Oct 30 19:30:11.220334 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:30:11.220312    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d79a6ed8c1722c23da5c1d85551a8da239f092ab4467b663ced62c7f662679ef\": container with ID starting with d79a6ed8c1722c23da5c1d85551a8da239f092ab4467b663ced62c7f662679ef not found: ID does not exist" containerID="d79a6ed8c1722c23da5c1d85551a8da239f092ab4467b663ced62c7f662679ef"
Oct 30 19:30:11.220411 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:11.220347    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d79a6ed8c1722c23da5c1d85551a8da239f092ab4467b663ced62c7f662679ef"} err="failed to get container status \"d79a6ed8c1722c23da5c1d85551a8da239f092ab4467b663ced62c7f662679ef\": rpc error: code = NotFound desc = could not find container \"d79a6ed8c1722c23da5c1d85551a8da239f092ab4467b663ced62c7f662679ef\": container with ID starting with d79a6ed8c1722c23da5c1d85551a8da239f092ab4467b663ced62c7f662679ef not found: ID does not exist"
Oct 30 19:30:11.369958 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:11.369925    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg"]
Oct 30 19:30:11.415906 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:11.415874    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-7d30a0-5df4b7mfpg"]
Oct 30 19:30:12.601451 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:12.601416    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="851f1425-8156-4269-9162-9ed1d56502e6" path="/var/lib/kubelet/pods/851f1425-8156-4269-9162-9ed1d56502e6/volumes"
Oct 30 19:30:35.530202 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:30:35.530154    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:30:37.605681 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:37.605645    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65"]
Oct 30 19:30:37.606517 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:37.605876    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65" podUID="d78e6841-0805-4163-a3cd-830833d24a2a" containerName="manager" containerID="cri-o://69f0cb4e4901e092eb71d4f327245754a8145f2fbb789d2424c6d0b2bb7f5b49" gracePeriod=10
Oct 30 19:30:37.858714 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:37.858661    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65"
Oct 30 19:30:38.003295 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:38.003266    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d78e6841-0805-4163-a3cd-830833d24a2a-kube-api-access-28jlz\" (UniqueName: \"kubernetes.io/projected/d78e6841-0805-4163-a3cd-830833d24a2a-kube-api-access-28jlz\") pod \"d78e6841-0805-4163-a3cd-830833d24a2a\" (UID: \"d78e6841-0805-4163-a3cd-830833d24a2a\") "
Oct 30 19:30:38.003447 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:38.003351    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d78e6841-0805-4163-a3cd-830833d24a2a-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d78e6841-0805-4163-a3cd-830833d24a2a-extensions-socket-volume\") pod \"d78e6841-0805-4163-a3cd-830833d24a2a\" (UID: \"d78e6841-0805-4163-a3cd-830833d24a2a\") "
Oct 30 19:30:38.003762 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:38.003736    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d78e6841-0805-4163-a3cd-830833d24a2a-extensions-socket-volume" pod "d78e6841-0805-4163-a3cd-830833d24a2a" (UID: "d78e6841-0805-4163-a3cd-830833d24a2a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:38.005231 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:38.005210    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d78e6841-0805-4163-a3cd-830833d24a2a-kube-api-access-28jlz" pod "d78e6841-0805-4163-a3cd-830833d24a2a" (UID: "d78e6841-0805-4163-a3cd-830833d24a2a"). InnerVolumeSpecName "kube-api-access-28jlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:38.104559 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:38.104522    4544 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d78e6841-0805-4163-a3cd-830833d24a2a-extensions-socket-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:38.104559 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:38.104552    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28jlz\" (UniqueName: \"kubernetes.io/projected/d78e6841-0805-4163-a3cd-830833d24a2a-kube-api-access-28jlz\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:38.357712 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:38.357675    4544 generic.go:356] "Generic (PLEG): container finished" podID="d78e6841-0805-4163-a3cd-830833d24a2a" containerID="69f0cb4e4901e092eb71d4f327245754a8145f2fbb789d2424c6d0b2bb7f5b49" exitCode=0
Oct 30 19:30:38.357886 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:38.357730    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65"
Oct 30 19:30:38.357886 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:38.357731    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65" event={"ID":"d78e6841-0805-4163-a3cd-830833d24a2a","Type":"ContainerDied","Data":"69f0cb4e4901e092eb71d4f327245754a8145f2fbb789d2424c6d0b2bb7f5b49"}
Oct 30 19:30:38.357886 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:38.357796    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65" event={"ID":"d78e6841-0805-4163-a3cd-830833d24a2a","Type":"ContainerDied","Data":"62134ca4732528cfb2e4bae4ccf55502e31fa54821cf1eeb92a61c7a3f8afe1e"}
Oct 30 19:30:38.357886 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:38.357829    4544 scope.go:122] "RemoveContainer" containerID="69f0cb4e4901e092eb71d4f327245754a8145f2fbb789d2424c6d0b2bb7f5b49"
Oct 30 19:30:38.368128 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:38.368108    4544 scope.go:122] "RemoveContainer" containerID="69f0cb4e4901e092eb71d4f327245754a8145f2fbb789d2424c6d0b2bb7f5b49"
Oct 30 19:30:38.368394 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:30:38.368377    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69f0cb4e4901e092eb71d4f327245754a8145f2fbb789d2424c6d0b2bb7f5b49\": container with ID starting with 69f0cb4e4901e092eb71d4f327245754a8145f2fbb789d2424c6d0b2bb7f5b49 not found: ID does not exist" containerID="69f0cb4e4901e092eb71d4f327245754a8145f2fbb789d2424c6d0b2bb7f5b49"
Oct 30 19:30:38.368461 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:38.368401    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69f0cb4e4901e092eb71d4f327245754a8145f2fbb789d2424c6d0b2bb7f5b49"} err="failed to get container status \"69f0cb4e4901e092eb71d4f327245754a8145f2fbb789d2424c6d0b2bb7f5b49\": rpc error: code = NotFound desc = could not find container \"69f0cb4e4901e092eb71d4f327245754a8145f2fbb789d2424c6d0b2bb7f5b49\": container with ID starting with 69f0cb4e4901e092eb71d4f327245754a8145f2fbb789d2424c6d0b2bb7f5b49 not found: ID does not exist"
Oct 30 19:30:38.553930 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:38.553890    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65"]
Oct 30 19:30:38.601673 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:38.601644    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-w6t65"]
Oct 30 19:30:40.600770 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:30:40.600736    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d78e6841-0805-4163-a3cd-830833d24a2a" path="/var/lib/kubelet/pods/d78e6841-0805-4163-a3cd-830833d24a2a/volumes"
Oct 30 19:31:05.530388 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:31:05.530352    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:31:35.530038 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:31:35.529999    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:31:41.262213 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.262179    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-plgsv"]
Oct 30 19:31:41.262687 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.262543    4544 cpu_manager.go:414] "containerMap: removing container" podUID="baaa260b-f29d-4e58-b900-d8bd50c382d0" containerName="istio-proxy"
Oct 30 19:31:41.262687 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.262553    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="baaa260b-f29d-4e58-b900-d8bd50c382d0" containerName="istio-proxy"
Oct 30 19:31:41.262687 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.262570    4544 cpu_manager.go:414] "containerMap: removing container" podUID="baaa260b-f29d-4e58-b900-d8bd50c382d0" containerName="istio-proxy"
Oct 30 19:31:41.262687 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.262575    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="baaa260b-f29d-4e58-b900-d8bd50c382d0" containerName="istio-proxy"
Oct 30 19:31:41.262687 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.262582    4544 cpu_manager.go:414] "containerMap: removing container" podUID="d78e6841-0805-4163-a3cd-830833d24a2a" containerName="manager"
Oct 30 19:31:41.262687 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.262589    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d78e6841-0805-4163-a3cd-830833d24a2a" containerName="manager"
Oct 30 19:31:41.262687 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.262623    4544 cpu_manager.go:414] "containerMap: removing container" podUID="47e6daf7-cd4f-4688-a878-194fc432f2fc" containerName="maas-api"
Oct 30 19:31:41.262687 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.262629    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47e6daf7-cd4f-4688-a878-194fc432f2fc" containerName="maas-api"
Oct 30 19:31:41.262687 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.262637    4544 cpu_manager.go:414] "containerMap: removing container" podUID="851f1425-8156-4269-9162-9ed1d56502e6" containerName="payload-pre-processing"
Oct 30 19:31:41.262687 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.262642    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="851f1425-8156-4269-9162-9ed1d56502e6" containerName="payload-pre-processing"
Oct 30 19:31:41.263004 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.262696    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="baaa260b-f29d-4e58-b900-d8bd50c382d0" containerName="istio-proxy"
Oct 30 19:31:41.263004 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.262705    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="d78e6841-0805-4163-a3cd-830833d24a2a" containerName="manager"
Oct 30 19:31:41.263004 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.262712    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="47e6daf7-cd4f-4688-a878-194fc432f2fc" containerName="maas-api"
Oct 30 19:31:41.263004 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.262717    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="851f1425-8156-4269-9162-9ed1d56502e6" containerName="payload-pre-processing"
Oct 30 19:31:41.265655 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.265622    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-plgsv"
Oct 30 19:31:41.281551 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.281524    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-plgsv"]
Oct 30 19:31:41.325981 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.325949    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wrjw5\""
Oct 30 19:31:41.358937 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.358904    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfgxk\" (UniqueName: \"kubernetes.io/projected/15e09f1b-51d2-4943-b382-d40f12efb9a0-kube-api-access-pfgxk\") pod \"kuadrant-operator-controller-manager-57bb9bfff-plgsv\" (UID: \"15e09f1b-51d2-4943-b382-d40f12efb9a0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-plgsv"
Oct 30 19:31:41.359081 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.359013    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15e09f1b-51d2-4943-b382-d40f12efb9a0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-plgsv\" (UID: \"15e09f1b-51d2-4943-b382-d40f12efb9a0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-plgsv"
Oct 30 19:31:41.459621 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.459576    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15e09f1b-51d2-4943-b382-d40f12efb9a0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-plgsv\" (UID: \"15e09f1b-51d2-4943-b382-d40f12efb9a0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-plgsv"
Oct 30 19:31:41.459782 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.459663    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfgxk\" (UniqueName: \"kubernetes.io/projected/15e09f1b-51d2-4943-b382-d40f12efb9a0-kube-api-access-pfgxk\") pod \"kuadrant-operator-controller-manager-57bb9bfff-plgsv\" (UID: \"15e09f1b-51d2-4943-b382-d40f12efb9a0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-plgsv"
Oct 30 19:31:41.460034 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.460014    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15e09f1b-51d2-4943-b382-d40f12efb9a0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-plgsv\" (UID: \"15e09f1b-51d2-4943-b382-d40f12efb9a0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-plgsv"
Oct 30 19:31:41.517526 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.517492    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfgxk\" (UniqueName: \"kubernetes.io/projected/15e09f1b-51d2-4943-b382-d40f12efb9a0-kube-api-access-pfgxk\") pod \"kuadrant-operator-controller-manager-57bb9bfff-plgsv\" (UID: \"15e09f1b-51d2-4943-b382-d40f12efb9a0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-plgsv"
Oct 30 19:31:41.577391 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:41.577358    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-plgsv"
Oct 30 19:31:42.049783 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:31:42.049740    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15e09f1b_51d2_4943_b382_d40f12efb9a0.slice/crio-6a4431f1ec093b77c5ed48654b1e2f9aaa22d1c333b6a3c9ff6ffa52d7e2b8e5 WatchSource:0}: Error finding container 6a4431f1ec093b77c5ed48654b1e2f9aaa22d1c333b6a3c9ff6ffa52d7e2b8e5: Status 404 returned error can't find the container with id 6a4431f1ec093b77c5ed48654b1e2f9aaa22d1c333b6a3c9ff6ffa52d7e2b8e5
Oct 30 19:31:42.086382 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:42.086354    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-plgsv"]
Oct 30 19:31:42.638375 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:42.638338    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-plgsv" event={"ID":"15e09f1b-51d2-4943-b382-d40f12efb9a0","Type":"ContainerStarted","Data":"dc83cd71f45a66eabe0c910ba8582b17202dd3e3ed791ee0c6cd19db22e7d798"}
Oct 30 19:31:42.638878 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:42.638394    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-plgsv" event={"ID":"15e09f1b-51d2-4943-b382-d40f12efb9a0","Type":"ContainerStarted","Data":"6a4431f1ec093b77c5ed48654b1e2f9aaa22d1c333b6a3c9ff6ffa52d7e2b8e5"}
Oct 30 19:31:42.638878 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:42.638431    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-plgsv"
Oct 30 19:31:42.844506 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:42.844436    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-plgsv" podStartSLOduration=1.844416144 podStartE2EDuration="1.844416144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:42.796845539 +0000 UTC m=+3300.322720635" watchObservedRunningTime="2026-10-30 19:31:42.844416144 +0000 UTC m=+3300.370291241"
Oct 30 19:31:43.037348 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:43.037318    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 19:31:43.041506 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:43.041484    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 19:31:53.645669 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:31:53.645633    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-plgsv"
Oct 30 19:32:05.531322 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:32:05.531285    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:32:35.530152 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:32:35.530114    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:32:39.883554 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:39.883522    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"]
Oct 30 19:32:39.884005 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:39.883977    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="baaa260b-f29d-4e58-b900-d8bd50c382d0" containerName="istio-proxy"
Oct 30 19:32:39.886816 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:39.886795    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:39.938147 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:39.938088    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-34d64c-openshift-default-dockercfg-xvj2w\""
Oct 30 19:32:39.966251 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:39.966214    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-workload-certs\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:39.966251 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:39.966256    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/957ed7c6-2f64-4526-a042-49930d758d52-istio-podinfo\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:39.966524 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:39.966289    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-workload-socket\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:39.966524 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:39.966433    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/957ed7c6-2f64-4526-a042-49930d758d52-istio-token\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:39.966524 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:39.966471    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd2zk\" (UniqueName: \"kubernetes.io/projected/957ed7c6-2f64-4526-a042-49930d758d52-kube-api-access-wd2zk\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:39.966524 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:39.966508    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-credential-socket\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:39.966731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:39.966578    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/957ed7c6-2f64-4526-a042-49930d758d52-istiod-ca-cert\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:39.966731 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:39.966660    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-istio-envoy\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:39.966807 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:39.966730    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-istio-data\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:39.984871 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:39.984841    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"]
Oct 30 19:32:40.067425 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.067392    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-workload-certs\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.067425 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.067428    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/957ed7c6-2f64-4526-a042-49930d758d52-istio-podinfo\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.067693 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.067455    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-workload-socket\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.067693 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.067580    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/957ed7c6-2f64-4526-a042-49930d758d52-istio-token\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.067693 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.067644    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd2zk\" (UniqueName: \"kubernetes.io/projected/957ed7c6-2f64-4526-a042-49930d758d52-kube-api-access-wd2zk\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.067693 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.067676    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-credential-socket\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.067927 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.067710    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/957ed7c6-2f64-4526-a042-49930d758d52-istiod-ca-cert\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.067927 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.067755    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-istio-envoy\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.067927 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.067853    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-istio-data\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.068090 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.067923    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-workload-certs\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.068152 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.068095    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-credential-socket\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.068152 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.068118    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-workload-socket\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.068260 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.068245    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-istio-data\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.068501 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.068482    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/957ed7c6-2f64-4526-a042-49930d758d52-istiod-ca-cert\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.069943 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.069920    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-istio-envoy\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.070052 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.069952    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/957ed7c6-2f64-4526-a042-49930d758d52-istio-podinfo\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.122114 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.122088    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/957ed7c6-2f64-4526-a042-49930d758d52-istio-token\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.122235 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.122146    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd2zk\" (UniqueName: \"kubernetes.io/projected/957ed7c6-2f64-4526-a042-49930d758d52-kube-api-access-wd2zk\") pod \"e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") " pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.200748 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.200690    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:40.629501 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.629446    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"]
Oct 30 19:32:40.632164 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:32:40.632135    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod957ed7c6_2f64_4526_a042_49930d758d52.slice/crio-f0725a6ae183345c529e8aff274a9eab49be0c388544c7fc30c3c18d99f34a0c WatchSource:0}: Error finding container f0725a6ae183345c529e8aff274a9eab49be0c388544c7fc30c3c18d99f34a0c: Status 404 returned error can't find the container with id f0725a6ae183345c529e8aff274a9eab49be0c388544c7fc30c3c18d99f34a0c
Oct 30 19:32:40.634241 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.634213    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:32:40.634290 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.634274    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:32:40.634329 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.634300    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:32:40.903077 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.902983    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw" event={"ID":"957ed7c6-2f64-4526-a042-49930d758d52","Type":"ContainerStarted","Data":"0570c6487bd081bfddd55bf054274e4c85b566ff4f565fa289727bc415c8faa0"}
Oct 30 19:32:40.903077 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:40.903048    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw" event={"ID":"957ed7c6-2f64-4526-a042-49930d758d52","Type":"ContainerStarted","Data":"f0725a6ae183345c529e8aff274a9eab49be0c388544c7fc30c3c18d99f34a0c"}
Oct 30 19:32:41.014499 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:41.014450    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw" podStartSLOduration=2.014436902 podStartE2EDuration="2.014436902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:41.013857921 +0000 UTC m=+3358.539733018" watchObservedRunningTime="2026-10-30 19:32:41.014436902 +0000 UTC m=+3358.540311998"
Oct 30 19:32:41.201850 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:41.201767    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:41.207227 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:41.207202    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:41.907181 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:41.907143    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:41.908077 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:41.908056    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:32:47.407464 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:47.407421    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7"]
Oct 30 19:32:47.411303 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:47.411278    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7"
Oct 30 19:32:47.511400 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:47.511346    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-34d64c\""
Oct 30 19:32:47.511688 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:47.511355    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-97f4d5f1-dockercfg-cfj98\""
Oct 30 19:32:47.514125 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:47.514095    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7"]
Oct 30 19:32:47.546250 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:47.546211    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e112914-3ddb-4723-aadd-9c6619d6dcae-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7\" (UID: \"6e112914-3ddb-4723-aadd-9c6619d6dcae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7"
Oct 30 19:32:47.546473 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:47.546263    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vccwc\" (UniqueName: \"kubernetes.io/projected/6e112914-3ddb-4723-aadd-9c6619d6dcae-kube-api-access-vccwc\") pod \"payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7\" (UID: \"6e112914-3ddb-4723-aadd-9c6619d6dcae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7"
Oct 30 19:32:47.647645 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:47.647574    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e112914-3ddb-4723-aadd-9c6619d6dcae-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7\" (UID: \"6e112914-3ddb-4723-aadd-9c6619d6dcae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7"
Oct 30 19:32:47.647824 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:47.647665    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vccwc\" (UniqueName: \"kubernetes.io/projected/6e112914-3ddb-4723-aadd-9c6619d6dcae-kube-api-access-vccwc\") pod \"payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7\" (UID: \"6e112914-3ddb-4723-aadd-9c6619d6dcae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7"
Oct 30 19:32:47.648372 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:47.648349    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e112914-3ddb-4723-aadd-9c6619d6dcae-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7\" (UID: \"6e112914-3ddb-4723-aadd-9c6619d6dcae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7"
Oct 30 19:32:47.703022 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:47.702939    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vccwc\" (UniqueName: \"kubernetes.io/projected/6e112914-3ddb-4723-aadd-9c6619d6dcae-kube-api-access-vccwc\") pod \"payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7\" (UID: \"6e112914-3ddb-4723-aadd-9c6619d6dcae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7"
Oct 30 19:32:47.725639 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:47.725575    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7"
Oct 30 19:32:48.094499 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:48.094413    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7"]
Oct 30 19:32:48.096871 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:32:48.096841    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e112914_3ddb_4723_aadd_9c6619d6dcae.slice/crio-207ba7a8db245105f8138af01eb42e5d312256f1e70ac8e2df0db96edcf37ce4 WatchSource:0}: Error finding container 207ba7a8db245105f8138af01eb42e5d312256f1e70ac8e2df0db96edcf37ce4: Status 404 returned error can't find the container with id 207ba7a8db245105f8138af01eb42e5d312256f1e70ac8e2df0db96edcf37ce4
Oct 30 19:32:48.939653 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:48.939587    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7" event={"ID":"6e112914-3ddb-4723-aadd-9c6619d6dcae","Type":"ContainerStarted","Data":"0a0032b3dd9a406f0a3063e462033a9304dfc554703964a20c10730ed28eb458"}
Oct 30 19:32:48.940058 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:48.939663    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7" event={"ID":"6e112914-3ddb-4723-aadd-9c6619d6dcae","Type":"ContainerStarted","Data":"207ba7a8db245105f8138af01eb42e5d312256f1e70ac8e2df0db96edcf37ce4"}
Oct 30 19:32:48.940058 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:48.939684    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7"
Oct 30 19:32:49.047272 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:49.047214    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7" podStartSLOduration=2.047197796 podStartE2EDuration="2.047197796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:49.046076966 +0000 UTC m=+3366.571952063" watchObservedRunningTime="2026-10-30 19:32:49.047197796 +0000 UTC m=+3366.573072892"
Oct 30 19:32:50.238500 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:50.238467    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"]
Oct 30 19:32:50.241969 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:50.241949    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:32:50.290788 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:50.290754    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-34d64c-serving-cert\""
Oct 30 19:32:50.339805 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:50.339773    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"]
Oct 30 19:32:50.374468 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:50.374437    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8c230a7-a34d-49ce-96a9-ae74783a0db0-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd\" (UID: \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:32:50.374663 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:50.374644    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ksnz\" (UniqueName: \"kubernetes.io/projected/d8c230a7-a34d-49ce-96a9-ae74783a0db0-kube-api-access-8ksnz\") pod \"maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd\" (UID: \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:32:50.374746 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:50.374689    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8c230a7-a34d-49ce-96a9-ae74783a0db0-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd\" (UID: \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:32:50.374813 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:50.374785    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8c230a7-a34d-49ce-96a9-ae74783a0db0-metrics-certs\") pod \"maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd\" (UID: \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:32:50.475820 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:50.475783    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8c230a7-a34d-49ce-96a9-ae74783a0db0-metrics-certs\") pod \"maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd\" (UID: \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:32:50.475998 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:50.475834    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8c230a7-a34d-49ce-96a9-ae74783a0db0-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd\" (UID: \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:32:50.475998 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:50.475912    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ksnz\" (UniqueName: \"kubernetes.io/projected/d8c230a7-a34d-49ce-96a9-ae74783a0db0-kube-api-access-8ksnz\") pod \"maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd\" (UID: \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:32:50.475998 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:50.475934    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8c230a7-a34d-49ce-96a9-ae74783a0db0-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd\" (UID: \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:32:50.475998 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:32:50.475990    4544 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-serving-cert: secret "maas-api-e2e-worker-single-main-34d64c-serving-cert" not found
Oct 30 19:32:50.476155 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:32:50.476065    4544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8c230a7-a34d-49ce-96a9-ae74783a0db0-maas-api-tls podName:d8c230a7-a34d-49ce-96a9-ae74783a0db0 nodeName:}" failed. No retries permitted until 2026-10-30 19:32:50.97603712 +0000 UTC m=+3368.501912194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d8c230a7-a34d-49ce-96a9-ae74783a0db0-maas-api-tls") pod "maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd" (UID: "d8c230a7-a34d-49ce-96a9-ae74783a0db0") : secret "maas-api-e2e-worker-single-main-34d64c-serving-cert" not found
Oct 30 19:32:50.476635 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:50.476580    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8c230a7-a34d-49ce-96a9-ae74783a0db0-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd\" (UID: \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:32:50.478452 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:50.478425    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8c230a7-a34d-49ce-96a9-ae74783a0db0-metrics-certs\") pod \"maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd\" (UID: \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:32:50.529741 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:50.529714    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ksnz\" (UniqueName: \"kubernetes.io/projected/d8c230a7-a34d-49ce-96a9-ae74783a0db0-kube-api-access-8ksnz\") pod \"maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd\" (UID: \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:32:50.980016 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:50.979935    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8c230a7-a34d-49ce-96a9-ae74783a0db0-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd\" (UID: \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:32:50.982299 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:50.982270    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8c230a7-a34d-49ce-96a9-ae74783a0db0-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd\" (UID: \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:32:51.158009 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:51.157960    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:32:51.530551 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:51.530519    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"]
Oct 30 19:32:51.534220 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:32:51.534188    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8c230a7_a34d_49ce_96a9_ae74783a0db0.slice/crio-8131d9edf34b9f0c6b18b2d1d604b9b1005d4eaee2b64fcaa827acc864ec32ad WatchSource:0}: Error finding container 8131d9edf34b9f0c6b18b2d1d604b9b1005d4eaee2b64fcaa827acc864ec32ad: Status 404 returned error can't find the container with id 8131d9edf34b9f0c6b18b2d1d604b9b1005d4eaee2b64fcaa827acc864ec32ad
Oct 30 19:32:51.953249 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:51.953161    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd" event={"ID":"d8c230a7-a34d-49ce-96a9-ae74783a0db0","Type":"ContainerStarted","Data":"8131d9edf34b9f0c6b18b2d1d604b9b1005d4eaee2b64fcaa827acc864ec32ad"}
Oct 30 19:32:52.960236 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:52.960159    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd" event={"ID":"d8c230a7-a34d-49ce-96a9-ae74783a0db0","Type":"ContainerStarted","Data":"7dbd871c02f071f27552dd43c18fcf5099c46a17db64702900b978f3a3c75817"}
Oct 30 19:32:52.960681 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:52.960312    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:32:53.069029 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:53.068962    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd" podStartSLOduration=2.197829034 podStartE2EDuration="3.068941531s" totalImagesPullingTime="871.112497ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:53.067551906 +0000 UTC m=+3370.593427001" watchObservedRunningTime="2026-10-30 19:32:53.068941531 +0000 UTC m=+3370.594816629"
Oct 30 19:32:58.986446 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:58.986415    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:32:59.948049 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:32:59.948020    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7"
Oct 30 19:33:05.532179 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:33:05.532119    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:33:08.033884 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:08.033853    4544 generic.go:356] "Generic (PLEG): container finished" podID="957ed7c6-2f64-4526-a042-49930d758d52" containerID="0570c6487bd081bfddd55bf054274e4c85b566ff4f565fa289727bc415c8faa0" exitCode=0
Oct 30 19:33:08.034428 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:08.033931    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw" event={"ID":"957ed7c6-2f64-4526-a042-49930d758d52","Type":"ContainerDied","Data":"0570c6487bd081bfddd55bf054274e4c85b566ff4f565fa289727bc415c8faa0"}
Oct 30 19:33:08.034428 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:08.034367    4544 scope.go:122] "RemoveContainer" containerID="0570c6487bd081bfddd55bf054274e4c85b566ff4f565fa289727bc415c8faa0"
Oct 30 19:33:08.035498 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:08.035473    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:33:08.035568 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:08.035546    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:33:08.035621 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:08.035596    4544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:33:09.039882 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:09.039849    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw" event={"ID":"957ed7c6-2f64-4526-a042-49930d758d52","Type":"ContainerStarted","Data":"b12f1e9d1f6f002c550e9b5fc5ed618e30ddf4ff1b65213142db9cc56d285859"}
Oct 30 19:33:09.201682 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:09.201646    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:33:09.203018 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:09.202988    4544 patch_prober.go:29] interesting pod/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 19:33:09.203148 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:09.203050    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw" podUID="957ed7c6-2f64-4526-a042-49930d758d52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 19:33:10.201485 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:10.201450    4544 patch_prober.go:29] interesting pod/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 19:33:10.201978 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:10.201524    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw" podUID="957ed7c6-2f64-4526-a042-49930d758d52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 19:33:11.201248 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:11.201215    4544 patch_prober.go:29] interesting pod/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 19:33:11.201487 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:11.201265    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw" podUID="957ed7c6-2f64-4526-a042-49930d758d52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 19:33:11.907288 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:11.907253    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:33:12.201256 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:12.201173    4544 patch_prober.go:29] interesting pod/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 19:33:12.201256 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:12.201224    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw" podUID="957ed7c6-2f64-4526-a042-49930d758d52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 19:33:13.202098 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:13.202061    4544 patch_prober.go:29] interesting pod/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 19:33:13.202517 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:13.202113    4544 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw" podUID="957ed7c6-2f64-4526-a042-49930d758d52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 19:33:14.205863 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:14.205831    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:33:14.206995 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:33:14.206972    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:33:35.530241 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:33:35.530200    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:34:05.354266 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:05.354237    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7"]
Oct 30 19:34:05.536435 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:34:05.536391    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:34:09.104986 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:09.104953    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"]
Oct 30 19:34:09.105466 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:09.105175    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd" podUID="d8c230a7-a34d-49ce-96a9-ae74783a0db0" containerName="maas-api" containerID="cri-o://7dbd871c02f071f27552dd43c18fcf5099c46a17db64702900b978f3a3c75817" gracePeriod=30
Oct 30 19:34:10.348903 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:10.348853    4544 generic.go:356] "Generic (PLEG): container finished" podID="d8c230a7-a34d-49ce-96a9-ae74783a0db0" containerID="7dbd871c02f071f27552dd43c18fcf5099c46a17db64702900b978f3a3c75817" exitCode=0
Oct 30 19:34:10.349323 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:10.348923    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd" event={"ID":"d8c230a7-a34d-49ce-96a9-ae74783a0db0","Type":"ContainerDied","Data":"7dbd871c02f071f27552dd43c18fcf5099c46a17db64702900b978f3a3c75817"}
Oct 30 19:34:10.349323 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:10.348970    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd" event={"ID":"d8c230a7-a34d-49ce-96a9-ae74783a0db0","Type":"ContainerDied","Data":"8131d9edf34b9f0c6b18b2d1d604b9b1005d4eaee2b64fcaa827acc864ec32ad"}
Oct 30 19:34:10.349323 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:10.348994    4544 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8131d9edf34b9f0c6b18b2d1d604b9b1005d4eaee2b64fcaa827acc864ec32ad"
Oct 30 19:34:10.355315 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:10.355283    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7" podUID="6e112914-3ddb-4723-aadd-9c6619d6dcae" containerName="payload-pre-processing" containerID="cri-o://0a0032b3dd9a406f0a3063e462033a9304dfc554703964a20c10730ed28eb458" gracePeriod=25
Oct 30 19:34:10.355466 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:10.355436    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:34:10.474041 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:10.473965    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d8c230a7-a34d-49ce-96a9-ae74783a0db0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8c230a7-a34d-49ce-96a9-ae74783a0db0-metrics-certs\") pod \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\" (UID: \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\") "
Oct 30 19:34:10.474204 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:10.474047    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d8c230a7-a34d-49ce-96a9-ae74783a0db0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8c230a7-a34d-49ce-96a9-ae74783a0db0-maas-api-tls\") pod \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\" (UID: \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\") "
Oct 30 19:34:10.474204 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:10.474149    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8c230a7-a34d-49ce-96a9-ae74783a0db0-kube-api-access-8ksnz\" (UniqueName: \"kubernetes.io/projected/d8c230a7-a34d-49ce-96a9-ae74783a0db0-kube-api-access-8ksnz\") pod \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\" (UID: \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\") "
Oct 30 19:34:10.474332 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:10.474230    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d8c230a7-a34d-49ce-96a9-ae74783a0db0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8c230a7-a34d-49ce-96a9-ae74783a0db0-openshift-service-ca\") pod \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\" (UID: \"d8c230a7-a34d-49ce-96a9-ae74783a0db0\") "
Oct 30 19:34:10.474615 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:10.474572    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8c230a7-a34d-49ce-96a9-ae74783a0db0-openshift-service-ca" pod "d8c230a7-a34d-49ce-96a9-ae74783a0db0" (UID: "d8c230a7-a34d-49ce-96a9-ae74783a0db0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:34:10.476393 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:10.476360    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8c230a7-a34d-49ce-96a9-ae74783a0db0-kube-api-access-8ksnz" pod "d8c230a7-a34d-49ce-96a9-ae74783a0db0" (UID: "d8c230a7-a34d-49ce-96a9-ae74783a0db0"). InnerVolumeSpecName "kube-api-access-8ksnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:10.476673 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:10.476648    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8c230a7-a34d-49ce-96a9-ae74783a0db0-maas-api-tls" pod "d8c230a7-a34d-49ce-96a9-ae74783a0db0" (UID: "d8c230a7-a34d-49ce-96a9-ae74783a0db0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:34:10.476771 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:10.476686    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8c230a7-a34d-49ce-96a9-ae74783a0db0-metrics-certs" pod "d8c230a7-a34d-49ce-96a9-ae74783a0db0" (UID: "d8c230a7-a34d-49ce-96a9-ae74783a0db0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:34:10.576499 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:10.576444    4544 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8c230a7-a34d-49ce-96a9-ae74783a0db0-maas-api-tls\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:10.576499 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:10.576484    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ksnz\" (UniqueName: \"kubernetes.io/projected/d8c230a7-a34d-49ce-96a9-ae74783a0db0-kube-api-access-8ksnz\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:10.576499 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:10.576501    4544 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8c230a7-a34d-49ce-96a9-ae74783a0db0-openshift-service-ca\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:10.576819 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:10.576515    4544 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8c230a7-a34d-49ce-96a9-ae74783a0db0-metrics-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:11.354323 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:11.354292    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"
Oct 30 19:34:11.507888 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:11.507856    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"]
Oct 30 19:34:11.553966 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:11.553935    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-34d64c-5bf5f75f97-czfkd"]
Oct 30 19:34:12.601338 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:12.601298    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d8c230a7-a34d-49ce-96a9-ae74783a0db0" path="/var/lib/kubelet/pods/d8c230a7-a34d-49ce-96a9-ae74783a0db0/volumes"
Oct 30 19:34:18.394263 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:18.394229    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"]
Oct 30 19:34:18.397893 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:18.394447    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw" podUID="957ed7c6-2f64-4526-a042-49930d758d52" containerName="istio-proxy" containerID="cri-o://b12f1e9d1f6f002c550e9b5fc5ed618e30ddf4ff1b65213142db9cc56d285859" gracePeriod=30
Oct 30 19:34:19.945450 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:19.945411    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7" podUID="6e112914-3ddb-4723-aadd-9c6619d6dcae" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:34:23.636919 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.636889    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:34:23.798056 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.798017    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-workload-certs\") pod \"957ed7c6-2f64-4526-a042-49930d758d52\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") "
Oct 30 19:34:23.798224 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.798076    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/957ed7c6-2f64-4526-a042-49930d758d52-istio-token\" (UniqueName: \"kubernetes.io/projected/957ed7c6-2f64-4526-a042-49930d758d52-istio-token\") pod \"957ed7c6-2f64-4526-a042-49930d758d52\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") "
Oct 30 19:34:23.798224 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.798110    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-workload-socket\") pod \"957ed7c6-2f64-4526-a042-49930d758d52\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") "
Oct 30 19:34:23.798224 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.798161    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-istio-envoy\") pod \"957ed7c6-2f64-4526-a042-49930d758d52\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") "
Oct 30 19:34:23.798224 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.798209    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/957ed7c6-2f64-4526-a042-49930d758d52-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/957ed7c6-2f64-4526-a042-49930d758d52-istio-podinfo\") pod \"957ed7c6-2f64-4526-a042-49930d758d52\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") "
Oct 30 19:34:23.798464 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.798256    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/957ed7c6-2f64-4526-a042-49930d758d52-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/957ed7c6-2f64-4526-a042-49930d758d52-istiod-ca-cert\") pod \"957ed7c6-2f64-4526-a042-49930d758d52\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") "
Oct 30 19:34:23.798464 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.798311    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-workload-certs" pod "957ed7c6-2f64-4526-a042-49930d758d52" (UID: "957ed7c6-2f64-4526-a042-49930d758d52"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:23.798464 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.798327    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-istio-data\") pod \"957ed7c6-2f64-4526-a042-49930d758d52\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") "
Oct 30 19:34:23.798464 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.798364    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-credential-socket\") pod \"957ed7c6-2f64-4526-a042-49930d758d52\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") "
Oct 30 19:34:23.798464 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.798401    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/957ed7c6-2f64-4526-a042-49930d758d52-kube-api-access-wd2zk\" (UniqueName: \"kubernetes.io/projected/957ed7c6-2f64-4526-a042-49930d758d52-kube-api-access-wd2zk\") pod \"957ed7c6-2f64-4526-a042-49930d758d52\" (UID: \"957ed7c6-2f64-4526-a042-49930d758d52\") "
Oct 30 19:34:23.799509 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.798719    4544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-workload-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:23.799509 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.798876    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/957ed7c6-2f64-4526-a042-49930d758d52-istiod-ca-cert" pod "957ed7c6-2f64-4526-a042-49930d758d52" (UID: "957ed7c6-2f64-4526-a042-49930d758d52"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:34:23.799509 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.799066    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-istio-data" pod "957ed7c6-2f64-4526-a042-49930d758d52" (UID: "957ed7c6-2f64-4526-a042-49930d758d52"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:23.799509 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.799262    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-workload-socket" pod "957ed7c6-2f64-4526-a042-49930d758d52" (UID: "957ed7c6-2f64-4526-a042-49930d758d52"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:23.799509 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.799327    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-credential-socket" pod "957ed7c6-2f64-4526-a042-49930d758d52" (UID: "957ed7c6-2f64-4526-a042-49930d758d52"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:23.801120 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.801096    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-istio-envoy" pod "957ed7c6-2f64-4526-a042-49930d758d52" (UID: "957ed7c6-2f64-4526-a042-49930d758d52"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:23.801302 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.801282    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/957ed7c6-2f64-4526-a042-49930d758d52-kube-api-access-wd2zk" pod "957ed7c6-2f64-4526-a042-49930d758d52" (UID: "957ed7c6-2f64-4526-a042-49930d758d52"). InnerVolumeSpecName "kube-api-access-wd2zk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:23.801499 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.801464    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/957ed7c6-2f64-4526-a042-49930d758d52-istio-podinfo" pod "957ed7c6-2f64-4526-a042-49930d758d52" (UID: "957ed7c6-2f64-4526-a042-49930d758d52"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:34:23.801499 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.801479    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/957ed7c6-2f64-4526-a042-49930d758d52-istio-token" pod "957ed7c6-2f64-4526-a042-49930d758d52" (UID: "957ed7c6-2f64-4526-a042-49930d758d52"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:23.900036 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.900008    4544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-istio-envoy\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:23.900036 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.900033    4544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/957ed7c6-2f64-4526-a042-49930d758d52-istio-podinfo\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:23.900036 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.900043    4544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/957ed7c6-2f64-4526-a042-49930d758d52-istiod-ca-cert\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:23.900329 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.900052    4544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-istio-data\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:23.900329 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.900062    4544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-credential-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:23.900329 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.900071    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wd2zk\" (UniqueName: \"kubernetes.io/projected/957ed7c6-2f64-4526-a042-49930d758d52-kube-api-access-wd2zk\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:23.900329 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.900081    4544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/957ed7c6-2f64-4526-a042-49930d758d52-istio-token\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:23.900329 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:23.900089    4544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/957ed7c6-2f64-4526-a042-49930d758d52-workload-socket\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:24.428509 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.428471    4544 generic.go:356] "Generic (PLEG): container finished" podID="957ed7c6-2f64-4526-a042-49930d758d52" containerID="b12f1e9d1f6f002c550e9b5fc5ed618e30ddf4ff1b65213142db9cc56d285859" exitCode=0
Oct 30 19:34:24.428736 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.428534    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"
Oct 30 19:34:24.428736 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.428575    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw" event={"ID":"957ed7c6-2f64-4526-a042-49930d758d52","Type":"ContainerDied","Data":"b12f1e9d1f6f002c550e9b5fc5ed618e30ddf4ff1b65213142db9cc56d285859"}
Oct 30 19:34:24.428736 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.428651    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw" event={"ID":"957ed7c6-2f64-4526-a042-49930d758d52","Type":"ContainerDied","Data":"f0725a6ae183345c529e8aff274a9eab49be0c388544c7fc30c3c18d99f34a0c"}
Oct 30 19:34:24.428736 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.428679    4544 scope.go:122] "RemoveContainer" containerID="b12f1e9d1f6f002c550e9b5fc5ed618e30ddf4ff1b65213142db9cc56d285859"
Oct 30 19:34:24.438378 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.438356    4544 scope.go:122] "RemoveContainer" containerID="0570c6487bd081bfddd55bf054274e4c85b566ff4f565fa289727bc415c8faa0"
Oct 30 19:34:24.445680 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.445662    4544 scope.go:122] "RemoveContainer" containerID="b12f1e9d1f6f002c550e9b5fc5ed618e30ddf4ff1b65213142db9cc56d285859"
Oct 30 19:34:24.445935 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:34:24.445910    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b12f1e9d1f6f002c550e9b5fc5ed618e30ddf4ff1b65213142db9cc56d285859\": container with ID starting with b12f1e9d1f6f002c550e9b5fc5ed618e30ddf4ff1b65213142db9cc56d285859 not found: ID does not exist" containerID="b12f1e9d1f6f002c550e9b5fc5ed618e30ddf4ff1b65213142db9cc56d285859"
Oct 30 19:34:24.445998 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.445940    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b12f1e9d1f6f002c550e9b5fc5ed618e30ddf4ff1b65213142db9cc56d285859"} err="failed to get container status \"b12f1e9d1f6f002c550e9b5fc5ed618e30ddf4ff1b65213142db9cc56d285859\": rpc error: code = NotFound desc = could not find container \"b12f1e9d1f6f002c550e9b5fc5ed618e30ddf4ff1b65213142db9cc56d285859\": container with ID starting with b12f1e9d1f6f002c550e9b5fc5ed618e30ddf4ff1b65213142db9cc56d285859 not found: ID does not exist"
Oct 30 19:34:24.445998 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.445957    4544 scope.go:122] "RemoveContainer" containerID="0570c6487bd081bfddd55bf054274e4c85b566ff4f565fa289727bc415c8faa0"
Oct 30 19:34:24.446191 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:34:24.446171    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0570c6487bd081bfddd55bf054274e4c85b566ff4f565fa289727bc415c8faa0\": container with ID starting with 0570c6487bd081bfddd55bf054274e4c85b566ff4f565fa289727bc415c8faa0 not found: ID does not exist" containerID="0570c6487bd081bfddd55bf054274e4c85b566ff4f565fa289727bc415c8faa0"
Oct 30 19:34:24.446235 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.446198    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0570c6487bd081bfddd55bf054274e4c85b566ff4f565fa289727bc415c8faa0"} err="failed to get container status \"0570c6487bd081bfddd55bf054274e4c85b566ff4f565fa289727bc415c8faa0\": rpc error: code = NotFound desc = could not find container \"0570c6487bd081bfddd55bf054274e4c85b566ff4f565fa289727bc415c8faa0\": container with ID starting with 0570c6487bd081bfddd55bf054274e4c85b566ff4f565fa289727bc415c8faa0 not found: ID does not exist"
Oct 30 19:34:24.631137 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.631103    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"]
Oct 30 19:34:24.677258 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.677227    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-34d64c-openshift-default-6c89b654fcqwvlw"]
Oct 30 19:34:24.683516 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.683457    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"]
Oct 30 19:34:24.683849 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.683835    4544 cpu_manager.go:414] "containerMap: removing container" podUID="d8c230a7-a34d-49ce-96a9-ae74783a0db0" containerName="maas-api"
Oct 30 19:34:24.683897 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.683850    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d8c230a7-a34d-49ce-96a9-ae74783a0db0" containerName="maas-api"
Oct 30 19:34:24.683897 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.683872    4544 cpu_manager.go:414] "containerMap: removing container" podUID="957ed7c6-2f64-4526-a042-49930d758d52" containerName="istio-proxy"
Oct 30 19:34:24.683897 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.683878    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="957ed7c6-2f64-4526-a042-49930d758d52" containerName="istio-proxy"
Oct 30 19:34:24.683897 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.683884    4544 cpu_manager.go:414] "containerMap: removing container" podUID="957ed7c6-2f64-4526-a042-49930d758d52" containerName="istio-proxy"
Oct 30 19:34:24.683897 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.683889    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="957ed7c6-2f64-4526-a042-49930d758d52" containerName="istio-proxy"
Oct 30 19:34:24.684051 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.683956    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="d8c230a7-a34d-49ce-96a9-ae74783a0db0" containerName="maas-api"
Oct 30 19:34:24.684051 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.683966    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="957ed7c6-2f64-4526-a042-49930d758d52" containerName="istio-proxy"
Oct 30 19:34:24.684051 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.683973    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="957ed7c6-2f64-4526-a042-49930d758d52" containerName="istio-proxy"
Oct 30 19:34:24.688459 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.688442    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:24.737677 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.737650    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-6a305904-kserve-self-signed-certs\""
Oct 30 19:34:24.743531 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.743494    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"]
Oct 30 19:34:24.809276 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.809247    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-model-cache\") pod \"e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") " pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:24.809435 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.809306    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-home\") pod \"e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") " pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:24.809435 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.809400    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61961a63-2629-4fca-b258-55e776a4f413-tls-certs\") pod \"e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") " pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:24.809435 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.809426    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-dshm\") pod \"e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") " pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:24.809541 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.809444    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79qtj\" (UniqueName: \"kubernetes.io/projected/61961a63-2629-4fca-b258-55e776a4f413-kube-api-access-79qtj\") pod \"e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") " pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:24.809594 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.809573    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-tmp-dir\") pod \"e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") " pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:24.910952 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.910913    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-tmp-dir\") pod \"e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") " pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:24.911126 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.910963    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-model-cache\") pod \"e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") " pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:24.911169 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.911141    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-home\") pod \"e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") " pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:24.911254 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.911237    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61961a63-2629-4fca-b258-55e776a4f413-tls-certs\") pod \"e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") " pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:24.911295 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.911270    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-dshm\") pod \"e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") " pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:24.911341 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.911290    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79qtj\" (UniqueName: \"kubernetes.io/projected/61961a63-2629-4fca-b258-55e776a4f413-kube-api-access-79qtj\") pod \"e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") " pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:24.911341 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.911317    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-model-cache\") pod \"e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") " pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:24.911341 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.911331    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-tmp-dir\") pod \"e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") " pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:24.911517 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.911500    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-home\") pod \"e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") " pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:24.913548 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.913519    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-dshm\") pod \"e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") " pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:24.913829 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.913810    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61961a63-2629-4fca-b258-55e776a4f413-tls-certs\") pod \"e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") " pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:24.965921 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:24.965843    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79qtj\" (UniqueName: \"kubernetes.io/projected/61961a63-2629-4fca-b258-55e776a4f413-kube-api-access-79qtj\") pod \"e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") " pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:25.009558 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:25.009519    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:25.385178 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:25.385150    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"]
Oct 30 19:34:25.387811 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:34:25.387785    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61961a63_2629_4fca_b258_55e776a4f413.slice/crio-d32e54c1c671a055bc25f6be8e28b12a07debe3b933f243c3d632806754a4b65 WatchSource:0}: Error finding container d32e54c1c671a055bc25f6be8e28b12a07debe3b933f243c3d632806754a4b65: Status 404 returned error can't find the container with id d32e54c1c671a055bc25f6be8e28b12a07debe3b933f243c3d632806754a4b65
Oct 30 19:34:25.434889 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:25.434852    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g" event={"ID":"61961a63-2629-4fca-b258-55e776a4f413","Type":"ContainerStarted","Data":"d32e54c1c671a055bc25f6be8e28b12a07debe3b933f243c3d632806754a4b65"}
Oct 30 19:34:26.441245 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:26.441200    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g" event={"ID":"61961a63-2629-4fca-b258-55e776a4f413","Type":"ContainerStarted","Data":"eea9d3a30a9dfdd99eeb663400247a8453e55360df537fde8fc74921a406a90c"}
Oct 30 19:34:26.550762 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:26.550687    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g" podStartSLOduration=2.5506681049999997 podStartE2EDuration="2.550668105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:34:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:34:26.550110755 +0000 UTC m=+3464.075985851" watchObservedRunningTime="2026-10-30 19:34:26.550668105 +0000 UTC m=+3464.076543202"
Oct 30 19:34:26.601747 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:26.601713    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="957ed7c6-2f64-4526-a042-49930d758d52" path="/var/lib/kubelet/pods/957ed7c6-2f64-4526-a042-49930d758d52/volumes"
Oct 30 19:34:27.010030 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:27.009997    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:29.945903 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:29.945856    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7" podUID="6e112914-3ddb-4723-aadd-9c6619d6dcae" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:34:30.992849 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:30.992812    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7"
Oct 30 19:34:31.123154 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:31.122677    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6e112914-3ddb-4723-aadd-9c6619d6dcae-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e112914-3ddb-4723-aadd-9c6619d6dcae-plugins-config-volume\") pod \"6e112914-3ddb-4723-aadd-9c6619d6dcae\" (UID: \"6e112914-3ddb-4723-aadd-9c6619d6dcae\") "
Oct 30 19:34:31.123484 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:31.123459    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e112914-3ddb-4723-aadd-9c6619d6dcae-kube-api-access-vccwc\" (UniqueName: \"kubernetes.io/projected/6e112914-3ddb-4723-aadd-9c6619d6dcae-kube-api-access-vccwc\") pod \"6e112914-3ddb-4723-aadd-9c6619d6dcae\" (UID: \"6e112914-3ddb-4723-aadd-9c6619d6dcae\") "
Oct 30 19:34:31.124081 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:31.123114    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e112914-3ddb-4723-aadd-9c6619d6dcae-plugins-config-volume" pod "6e112914-3ddb-4723-aadd-9c6619d6dcae" (UID: "6e112914-3ddb-4723-aadd-9c6619d6dcae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:34:31.124642 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:31.124622    4544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e112914-3ddb-4723-aadd-9c6619d6dcae-plugins-config-volume\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:31.140295 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:31.140251    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e112914-3ddb-4723-aadd-9c6619d6dcae-kube-api-access-vccwc" pod "6e112914-3ddb-4723-aadd-9c6619d6dcae" (UID: "6e112914-3ddb-4723-aadd-9c6619d6dcae"). InnerVolumeSpecName "kube-api-access-vccwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:31.228340 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:31.228256    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vccwc\" (UniqueName: \"kubernetes.io/projected/6e112914-3ddb-4723-aadd-9c6619d6dcae-kube-api-access-vccwc\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:31.496854 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:31.496760    4544 generic.go:356] "Generic (PLEG): container finished" podID="6e112914-3ddb-4723-aadd-9c6619d6dcae" containerID="0a0032b3dd9a406f0a3063e462033a9304dfc554703964a20c10730ed28eb458" exitCode=0
Oct 30 19:34:31.497165 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:31.497139    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7" event={"ID":"6e112914-3ddb-4723-aadd-9c6619d6dcae","Type":"ContainerDied","Data":"0a0032b3dd9a406f0a3063e462033a9304dfc554703964a20c10730ed28eb458"}
Oct 30 19:34:31.497344 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:31.497317    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7" event={"ID":"6e112914-3ddb-4723-aadd-9c6619d6dcae","Type":"ContainerDied","Data":"207ba7a8db245105f8138af01eb42e5d312256f1e70ac8e2df0db96edcf37ce4"}
Oct 30 19:34:31.497482 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:31.497468    4544 scope.go:122] "RemoveContainer" containerID="0a0032b3dd9a406f0a3063e462033a9304dfc554703964a20c10730ed28eb458"
Oct 30 19:34:31.501826 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:31.501804    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7"
Oct 30 19:34:31.532850 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:31.532822    4544 scope.go:122] "RemoveContainer" containerID="0a0032b3dd9a406f0a3063e462033a9304dfc554703964a20c10730ed28eb458"
Oct 30 19:34:31.533418 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:34:31.533389    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a0032b3dd9a406f0a3063e462033a9304dfc554703964a20c10730ed28eb458\": container with ID starting with 0a0032b3dd9a406f0a3063e462033a9304dfc554703964a20c10730ed28eb458 not found: ID does not exist" containerID="0a0032b3dd9a406f0a3063e462033a9304dfc554703964a20c10730ed28eb458"
Oct 30 19:34:31.533535 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:31.533428    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a0032b3dd9a406f0a3063e462033a9304dfc554703964a20c10730ed28eb458"} err="failed to get container status \"0a0032b3dd9a406f0a3063e462033a9304dfc554703964a20c10730ed28eb458\": rpc error: code = NotFound desc = could not find container \"0a0032b3dd9a406f0a3063e462033a9304dfc554703964a20c10730ed28eb458\": container with ID starting with 0a0032b3dd9a406f0a3063e462033a9304dfc554703964a20c10730ed28eb458 not found: ID does not exist"
Oct 30 19:34:31.733196 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:31.733148    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7"]
Oct 30 19:34:31.733375 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:31.733227    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-34d64c-6f568blzw7"]
Oct 30 19:34:32.602329 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:32.602298    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e112914-3ddb-4723-aadd-9c6619d6dcae" path="/var/lib/kubelet/pods/6e112914-3ddb-4723-aadd-9c6619d6dcae/volumes"
Oct 30 19:34:35.010059 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:35.010025    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:35.022391 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:35.022365    4544 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:35.528550 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:34:35.528521    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:34:35.531682 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:34:35.531653    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:35:05.532384 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:35:05.532335    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:35:29.131359 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:29.131323    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"]
Oct 30 19:35:35.635921 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:35:35.635878    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:35:44.150812 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.150753    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g" podUID="61961a63-2629-4fca-b258-55e776a4f413" containerName="main" containerID="cri-o://eea9d3a30a9dfdd99eeb663400247a8453e55360df537fde8fc74921a406a90c" gracePeriod=45
Oct 30 19:35:44.522722 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.516060    4544 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g" podUID="61961a63-2629-4fca-b258-55e776a4f413" containerName="main" probeResult="failure" output="Get \"https://10.133.0.136:8000/ready\": dial tcp 10.133.0.136:8000: connect: connection refused"
Oct 30 19:35:44.837791 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.837709    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:35:44.928717 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.928680    4544 generic.go:356] "Generic (PLEG): container finished" podID="61961a63-2629-4fca-b258-55e776a4f413" containerID="eea9d3a30a9dfdd99eeb663400247a8453e55360df537fde8fc74921a406a90c" exitCode=0
Oct 30 19:35:44.929014 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.928990    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g" event={"ID":"61961a63-2629-4fca-b258-55e776a4f413","Type":"ContainerDied","Data":"eea9d3a30a9dfdd99eeb663400247a8453e55360df537fde8fc74921a406a90c"}
Oct 30 19:35:44.929163 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.929144    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g" event={"ID":"61961a63-2629-4fca-b258-55e776a4f413","Type":"ContainerDied","Data":"d32e54c1c671a055bc25f6be8e28b12a07debe3b933f243c3d632806754a4b65"}
Oct 30 19:35:44.929277 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.929262    4544 scope.go:122] "RemoveContainer" containerID="eea9d3a30a9dfdd99eeb663400247a8453e55360df537fde8fc74921a406a90c"
Oct 30 19:35:44.929540 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.929522    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"
Oct 30 19:35:44.940715 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.940661    4544 scope.go:122] "RemoveContainer" containerID="eea9d3a30a9dfdd99eeb663400247a8453e55360df537fde8fc74921a406a90c"
Oct 30 19:35:44.941045 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:35:44.941026    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eea9d3a30a9dfdd99eeb663400247a8453e55360df537fde8fc74921a406a90c\": container with ID starting with eea9d3a30a9dfdd99eeb663400247a8453e55360df537fde8fc74921a406a90c not found: ID does not exist" containerID="eea9d3a30a9dfdd99eeb663400247a8453e55360df537fde8fc74921a406a90c"
Oct 30 19:35:44.941141 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.941089    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eea9d3a30a9dfdd99eeb663400247a8453e55360df537fde8fc74921a406a90c"} err="failed to get container status \"eea9d3a30a9dfdd99eeb663400247a8453e55360df537fde8fc74921a406a90c\": rpc error: code = NotFound desc = could not find container \"eea9d3a30a9dfdd99eeb663400247a8453e55360df537fde8fc74921a406a90c\": container with ID starting with eea9d3a30a9dfdd99eeb663400247a8453e55360df537fde8fc74921a406a90c not found: ID does not exist"
Oct 30 19:35:44.949929 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.949908    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-tmp-dir\") pod \"61961a63-2629-4fca-b258-55e776a4f413\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") "
Oct 30 19:35:44.950039 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.950028    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61961a63-2629-4fca-b258-55e776a4f413-tls-certs\" (UniqueName: \"kubernetes.io/secret/61961a63-2629-4fca-b258-55e776a4f413-tls-certs\") pod \"61961a63-2629-4fca-b258-55e776a4f413\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") "
Oct 30 19:35:44.950094 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.950048    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61961a63-2629-4fca-b258-55e776a4f413-kube-api-access-79qtj\" (UniqueName: \"kubernetes.io/projected/61961a63-2629-4fca-b258-55e776a4f413-kube-api-access-79qtj\") pod \"61961a63-2629-4fca-b258-55e776a4f413\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") "
Oct 30 19:35:44.950094 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.950080    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-home\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-home\") pod \"61961a63-2629-4fca-b258-55e776a4f413\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") "
Oct 30 19:35:44.950196 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.950096    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-dshm\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-dshm\") pod \"61961a63-2629-4fca-b258-55e776a4f413\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") "
Oct 30 19:35:44.950196 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.950129    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-model-cache\") pod \"61961a63-2629-4fca-b258-55e776a4f413\" (UID: \"61961a63-2629-4fca-b258-55e776a4f413\") "
Oct 30 19:35:44.950196 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.950157    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-tmp-dir" pod "61961a63-2629-4fca-b258-55e776a4f413" (UID: "61961a63-2629-4fca-b258-55e776a4f413"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:44.950359 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.950343    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-home" pod "61961a63-2629-4fca-b258-55e776a4f413" (UID: "61961a63-2629-4fca-b258-55e776a4f413"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:44.950460 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.950441    4544 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-home\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:44.950530 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.950464    4544 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-tmp-dir\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:44.950530 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.950517    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-model-cache" pod "61961a63-2629-4fca-b258-55e776a4f413" (UID: "61961a63-2629-4fca-b258-55e776a4f413"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:44.952292 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.952265    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-dshm" pod "61961a63-2629-4fca-b258-55e776a4f413" (UID: "61961a63-2629-4fca-b258-55e776a4f413"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:44.952407 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.952330    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61961a63-2629-4fca-b258-55e776a4f413-tls-certs" pod "61961a63-2629-4fca-b258-55e776a4f413" (UID: "61961a63-2629-4fca-b258-55e776a4f413"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:35:44.952407 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:44.952392    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61961a63-2629-4fca-b258-55e776a4f413-kube-api-access-79qtj" pod "61961a63-2629-4fca-b258-55e776a4f413" (UID: "61961a63-2629-4fca-b258-55e776a4f413"). InnerVolumeSpecName "kube-api-access-79qtj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:45.051414 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:45.051382    4544 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61961a63-2629-4fca-b258-55e776a4f413-tls-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:45.051414 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:45.051413    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79qtj\" (UniqueName: \"kubernetes.io/projected/61961a63-2629-4fca-b258-55e776a4f413-kube-api-access-79qtj\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:45.051414 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:45.051423    4544 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-dshm\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:45.051683 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:45.051432    4544 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61961a63-2629-4fca-b258-55e776a4f413-model-cache\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:45.371833 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:45.371657    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"]
Oct 30 19:35:45.376539 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:45.376506    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-6a305904-kserve-6c48f4846f-dqc6g"]
Oct 30 19:35:46.601071 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:35:46.601036    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61961a63-2629-4fca-b258-55e776a4f413" path="/var/lib/kubelet/pods/61961a63-2629-4fca-b258-55e776a4f413/volumes"
Oct 30 19:36:05.531623 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:36:05.531559    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:36:26.076093 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.076054    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"]
Oct 30 19:36:26.076651 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.076410    4544 cpu_manager.go:414] "containerMap: removing container" podUID="61961a63-2629-4fca-b258-55e776a4f413" containerName="main"
Oct 30 19:36:26.076651 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.076421    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61961a63-2629-4fca-b258-55e776a4f413" containerName="main"
Oct 30 19:36:26.076651 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.076444    4544 cpu_manager.go:414] "containerMap: removing container" podUID="6e112914-3ddb-4723-aadd-9c6619d6dcae" containerName="payload-pre-processing"
Oct 30 19:36:26.076651 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.076450    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e112914-3ddb-4723-aadd-9c6619d6dcae" containerName="payload-pre-processing"
Oct 30 19:36:26.076651 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.076519    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="61961a63-2629-4fca-b258-55e776a4f413" containerName="main"
Oct 30 19:36:26.076651 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.076530    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e112914-3ddb-4723-aadd-9c6619d6dcae" containerName="payload-pre-processing"
Oct 30 19:36:26.079521 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.079502    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"
Oct 30 19:36:26.170937 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.170910    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d-maas-api-tls\") pod \"maas-api-595cc648c-b9fk4\" (UID: \"79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"
Oct 30 19:36:26.171122 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.170958    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xhq5\" (UniqueName: \"kubernetes.io/projected/79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d-kube-api-access-4xhq5\") pod \"maas-api-595cc648c-b9fk4\" (UID: \"79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"
Oct 30 19:36:26.171122 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.171023    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d-metrics-certs\") pod \"maas-api-595cc648c-b9fk4\" (UID: \"79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"
Oct 30 19:36:26.171122 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.171042    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d-openshift-service-ca\") pod \"maas-api-595cc648c-b9fk4\" (UID: \"79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"
Oct 30 19:36:26.176767 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.176738    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"]
Oct 30 19:36:26.272195 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.272164    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d-maas-api-tls\") pod \"maas-api-595cc648c-b9fk4\" (UID: \"79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"
Oct 30 19:36:26.272371 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.272210    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xhq5\" (UniqueName: \"kubernetes.io/projected/79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d-kube-api-access-4xhq5\") pod \"maas-api-595cc648c-b9fk4\" (UID: \"79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"
Oct 30 19:36:26.272371 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.272229    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d-metrics-certs\") pod \"maas-api-595cc648c-b9fk4\" (UID: \"79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"
Oct 30 19:36:26.272371 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.272247    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d-openshift-service-ca\") pod \"maas-api-595cc648c-b9fk4\" (UID: \"79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"
Oct 30 19:36:26.272888 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.272864    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d-openshift-service-ca\") pod \"maas-api-595cc648c-b9fk4\" (UID: \"79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"
Oct 30 19:36:26.274656 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.274627    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d-metrics-certs\") pod \"maas-api-595cc648c-b9fk4\" (UID: \"79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"
Oct 30 19:36:26.274656 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.274641    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d-maas-api-tls\") pod \"maas-api-595cc648c-b9fk4\" (UID: \"79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"
Oct 30 19:36:26.325502 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.325469    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xhq5\" (UniqueName: \"kubernetes.io/projected/79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d-kube-api-access-4xhq5\") pod \"maas-api-595cc648c-b9fk4\" (UID: \"79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"
Oct 30 19:36:26.393108 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.393042    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"
Oct 30 19:36:26.758085 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.758062    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"]
Oct 30 19:36:26.760397 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:36:26.760366    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ca8d19_7b1d_4fbf_ab33_9f3e22176c4d.slice/crio-2ac7b40270cef1a279d6ccbffcc0568e7ad1f683a3a7354b905a5eddf17194ea WatchSource:0}: Error finding container 2ac7b40270cef1a279d6ccbffcc0568e7ad1f683a3a7354b905a5eddf17194ea: Status 404 returned error can't find the container with id 2ac7b40270cef1a279d6ccbffcc0568e7ad1f683a3a7354b905a5eddf17194ea
Oct 30 19:36:26.761593 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:26.761574    4544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:36:27.150434 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:27.150400    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4" event={"ID":"79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d","Type":"ContainerStarted","Data":"2ac7b40270cef1a279d6ccbffcc0568e7ad1f683a3a7354b905a5eddf17194ea"}
Oct 30 19:36:28.157364 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:28.157328    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4" event={"ID":"79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d","Type":"ContainerStarted","Data":"c322c60975aca2443985adaeae09079e520a80a32b437f49c2c3a602c71df266"}
Oct 30 19:36:28.157829 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:28.157477    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"
Oct 30 19:36:34.166852 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:34.166822    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4"
Oct 30 19:36:34.275906 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:34.275850    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-595cc648c-b9fk4" podStartSLOduration=7.478209237 podStartE2EDuration="8.275835671s" totalImagesPullingTime="797.626434ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:28.265072246 +0000 UTC m=+3585.790947355" watchObservedRunningTime="2026-10-30 19:36:34.275835671 +0000 UTC m=+3591.801710764"
Oct 30 19:36:34.323026 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:34.322994    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"]
Oct 30 19:36:34.323244 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:34.323219    4544 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj" podUID="5c426d1d-63bf-4145-99f0-7e08baef8a9e" containerName="maas-api" containerID="cri-o://035ab5891686a08dd1723141c69109df3889ff633369ba5c4ca247cd63fd08f7" gracePeriod=30
Oct 30 19:36:35.536787 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:36:35.536745    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:36:35.569063 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:35.569038    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"
Oct 30 19:36:35.656365 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:35.656336    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5c426d1d-63bf-4145-99f0-7e08baef8a9e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c426d1d-63bf-4145-99f0-7e08baef8a9e-metrics-certs\") pod \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\" (UID: \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\") "
Oct 30 19:36:35.656540 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:35.656379    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5c426d1d-63bf-4145-99f0-7e08baef8a9e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c426d1d-63bf-4145-99f0-7e08baef8a9e-maas-api-tls\") pod \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\" (UID: \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\") "
Oct 30 19:36:35.656540 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:35.656494    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c426d1d-63bf-4145-99f0-7e08baef8a9e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c426d1d-63bf-4145-99f0-7e08baef8a9e-openshift-service-ca\") pod \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\" (UID: \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\") "
Oct 30 19:36:35.656540 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:35.656536    4544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c426d1d-63bf-4145-99f0-7e08baef8a9e-kube-api-access-j5xtc\" (UniqueName: \"kubernetes.io/projected/5c426d1d-63bf-4145-99f0-7e08baef8a9e-kube-api-access-j5xtc\") pod \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\" (UID: \"5c426d1d-63bf-4145-99f0-7e08baef8a9e\") "
Oct 30 19:36:35.656900 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:35.656873    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c426d1d-63bf-4145-99f0-7e08baef8a9e-openshift-service-ca" pod "5c426d1d-63bf-4145-99f0-7e08baef8a9e" (UID: "5c426d1d-63bf-4145-99f0-7e08baef8a9e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:35.658628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:35.658550    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c426d1d-63bf-4145-99f0-7e08baef8a9e-metrics-certs" pod "5c426d1d-63bf-4145-99f0-7e08baef8a9e" (UID: "5c426d1d-63bf-4145-99f0-7e08baef8a9e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:35.658628 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:35.658599    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c426d1d-63bf-4145-99f0-7e08baef8a9e-maas-api-tls" pod "5c426d1d-63bf-4145-99f0-7e08baef8a9e" (UID: "5c426d1d-63bf-4145-99f0-7e08baef8a9e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:35.658756 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:35.658719    4544 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c426d1d-63bf-4145-99f0-7e08baef8a9e-kube-api-access-j5xtc" pod "5c426d1d-63bf-4145-99f0-7e08baef8a9e" (UID: "5c426d1d-63bf-4145-99f0-7e08baef8a9e"). InnerVolumeSpecName "kube-api-access-j5xtc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:35.758160 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:35.758112    4544 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c426d1d-63bf-4145-99f0-7e08baef8a9e-openshift-service-ca\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:35.758160 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:35.758155    4544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5xtc\" (UniqueName: \"kubernetes.io/projected/5c426d1d-63bf-4145-99f0-7e08baef8a9e-kube-api-access-j5xtc\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:35.758406 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:35.758171    4544 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c426d1d-63bf-4145-99f0-7e08baef8a9e-metrics-certs\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:35.758406 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:35.758187    4544 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c426d1d-63bf-4145-99f0-7e08baef8a9e-maas-api-tls\") on node \"ip-10-0-140-122.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:36.193206 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:36.193168    4544 generic.go:356] "Generic (PLEG): container finished" podID="5c426d1d-63bf-4145-99f0-7e08baef8a9e" containerID="035ab5891686a08dd1723141c69109df3889ff633369ba5c4ca247cd63fd08f7" exitCode=0
Oct 30 19:36:36.193206 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:36.193207    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj" event={"ID":"5c426d1d-63bf-4145-99f0-7e08baef8a9e","Type":"ContainerDied","Data":"035ab5891686a08dd1723141c69109df3889ff633369ba5c4ca247cd63fd08f7"}
Oct 30 19:36:36.193457 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:36.193220    4544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"
Oct 30 19:36:36.193457 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:36.193246    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-l22pj" event={"ID":"5c426d1d-63bf-4145-99f0-7e08baef8a9e","Type":"ContainerDied","Data":"49ad6305f328d8fc38f8cdd2bd4c7345893097a458dbf9e5f9ee3622a61946b1"}
Oct 30 19:36:36.193457 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:36.193270    4544 scope.go:122] "RemoveContainer" containerID="035ab5891686a08dd1723141c69109df3889ff633369ba5c4ca247cd63fd08f7"
Oct 30 19:36:36.202700 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:36.202683    4544 scope.go:122] "RemoveContainer" containerID="035ab5891686a08dd1723141c69109df3889ff633369ba5c4ca247cd63fd08f7"
Oct 30 19:36:36.202944 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:36:36.202926    4544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"035ab5891686a08dd1723141c69109df3889ff633369ba5c4ca247cd63fd08f7\": container with ID starting with 035ab5891686a08dd1723141c69109df3889ff633369ba5c4ca247cd63fd08f7 not found: ID does not exist" containerID="035ab5891686a08dd1723141c69109df3889ff633369ba5c4ca247cd63fd08f7"
Oct 30 19:36:36.203037 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:36.202951    4544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"035ab5891686a08dd1723141c69109df3889ff633369ba5c4ca247cd63fd08f7"} err="failed to get container status \"035ab5891686a08dd1723141c69109df3889ff633369ba5c4ca247cd63fd08f7\": rpc error: code = NotFound desc = could not find container \"035ab5891686a08dd1723141c69109df3889ff633369ba5c4ca247cd63fd08f7\": container with ID starting with 035ab5891686a08dd1723141c69109df3889ff633369ba5c4ca247cd63fd08f7 not found: ID does not exist"
Oct 30 19:36:36.354942 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:36.354914    4544 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"]
Oct 30 19:36:36.401122 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:36.401088    4544 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-l22pj"]
Oct 30 19:36:36.602019 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:36.601981    4544 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c426d1d-63bf-4145-99f0-7e08baef8a9e" path="/var/lib/kubelet/pods/5c426d1d-63bf-4145-99f0-7e08baef8a9e/volumes"
Oct 30 19:36:43.065651 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:43.065595    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 19:36:43.071845 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:36:43.071821    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxnnz_114fbcb8-1004-410d-98bc-dc60f8fcae3c/ovn-acl-logging/0.log"
Oct 30 19:37:05.530863 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:37:05.530824    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:37:35.530528 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:37:35.530493    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:37:51.851804 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:37:51.851764    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-c8f867457-rcrqp_ae711732-bab8-4403-9bf1-c73a98733597/manager/0.log"
Oct 30 19:37:53.674350 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:37:53.674315    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7496565bcf-r9dth_fb9ff216-796a-44c4-95ee-ce96085894f1/manager/0.log"
Oct 30 19:37:54.869924 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:37:54.869892    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs_e61c8a9d-9926-46cb-85e7-b704618d5348/copy-manifests/0.log"
Oct 30 19:37:54.946000 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:37:54.945968    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs_e61c8a9d-9926-46cb-85e7-b704618d5348/manager/0.log"
Oct 30 19:37:55.845404 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:37:55.845376    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-7d94f55f4b-jnwbh_2abf6a89-8238-4417-8f77-8346ca0b7780/proxy/0.log"
Oct 30 19:37:56.763806 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:37:56.763780    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-b9fk4_79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d/maas-api/0.log"
Oct 30 19:37:58.998977 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:37:58.998944    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-9zm7n_d2f0a477-b0d0-4779-9b17-4ce095ab1347/postgres/0.log"
Oct 30 19:38:01.157086 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:01.157055    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d6dd76dc-q9wt9_59f3ebf1-4b49-4789-bb7c-cca228cfc72a/manager/0.log"
Oct 30 19:38:01.985584 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:01.985556    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57bb9bfff-plgsv_15e09f1b-51d2-4943-b382-d40f12efb9a0/manager/0.log"
Oct 30 19:38:02.685355 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:02.685322    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-856dd584c5-khfdk_3684a66d-aa78-4d80-8229-4b32be8f7ac2/manager/0.log"
Oct 30 19:38:05.082236 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:05.082207    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75d7c785f-cjv6v_e269d837-0eb0-4d79-9c11-2f411871f859/istio-proxy/0.log"
Oct 30 19:38:05.380864 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:05.380789    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75d7c785f-g58p2_85189f79-bfea-4791-bad5-f30a93398ea8/istio-proxy/0.log"
Oct 30 19:38:05.532188 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:38:05.532151    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:38:06.367880 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:06.367851    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75d7c785f-nf2pw_28e7f033-4399-4555-8e0b-d2642bbf3837/istio-proxy/0.log"
Oct 30 19:38:08.487663 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:08.487632    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn_5bffeda4-a4da-4657-86c0-ec91bc1abd2b/main/0.log"
Oct 30 19:38:08.760515 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:08.760440    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-4hrz6_b247e514-74ee-4b71-8466-b184215bf490/main/0.log"
Oct 30 19:38:09.585075 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:09.585044    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc_7a3bc726-52a3-4679-b0ac-9910abdb651e/main/0.log"
Oct 30 19:38:21.045035 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:21.045004    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-c8f867457-rcrqp_ae711732-bab8-4403-9bf1-c73a98733597/manager/0.log"
Oct 30 19:38:22.317310 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:22.317272    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7496565bcf-r9dth_fb9ff216-796a-44c4-95ee-ce96085894f1/manager/0.log"
Oct 30 19:38:23.564553 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:23.564526    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs_e61c8a9d-9926-46cb-85e7-b704618d5348/copy-manifests/0.log"
Oct 30 19:38:23.640935 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:23.640905    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9c7b5ddd8-ctpvs_e61c8a9d-9926-46cb-85e7-b704618d5348/manager/0.log"
Oct 30 19:38:24.607919 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:24.607887    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-7d94f55f4b-jnwbh_2abf6a89-8238-4417-8f77-8346ca0b7780/proxy/0.log"
Oct 30 19:38:25.278216 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:25.278188    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-b9fk4_79ca8d19-7b1d-4fbf-ab33-9f3e22176c4d/maas-api/0.log"
Oct 30 19:38:27.496001 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:27.495971    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-9zm7n_d2f0a477-b0d0-4779-9b17-4ce095ab1347/postgres/0.log"
Oct 30 19:38:29.244694 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:29.244663    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d6dd76dc-q9wt9_59f3ebf1-4b49-4789-bb7c-cca228cfc72a/manager/0.log"
Oct 30 19:38:30.043794 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:30.043766    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57bb9bfff-plgsv_15e09f1b-51d2-4943-b382-d40f12efb9a0/manager/0.log"
Oct 30 19:38:30.729980 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:30.729952    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-856dd584c5-khfdk_3684a66d-aa78-4d80-8229-4b32be8f7ac2/manager/0.log"
Oct 30 19:38:32.914054 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:32.914023    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75d7c785f-cjv6v_e269d837-0eb0-4d79-9c11-2f411871f859/istio-proxy/0.log"
Oct 30 19:38:33.184413 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:33.184337    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75d7c785f-g58p2_85189f79-bfea-4791-bad5-f30a93398ea8/istio-proxy/0.log"
Oct 30 19:38:34.156722 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:34.156689    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75d7c785f-nf2pw_28e7f033-4399-4555-8e0b-d2642bbf3837/istio-proxy/0.log"
Oct 30 19:38:35.532421 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:38:35.532383    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:38:35.996050 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:35.995978    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-rskxn_5bffeda4-a4da-4657-86c0-ec91bc1abd2b/main/0.log"
Oct 30 19:38:36.285514 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:36.285485    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-4hrz6_b247e514-74ee-4b71-8466-b184215bf490/main/0.log"
Oct 30 19:38:37.103010 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:38:37.102974    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8qqkwc_7a3bc726-52a3-4679-b0ac-9910abdb651e/main/0.log"
Oct 30 19:39:05.534262 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:39:05.534179    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:39:08.803686 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:39:08.803657    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7496565bcf-r9dth_fb9ff216-796a-44c4-95ee-ce96085894f1/manager/0.log"
Oct 30 19:39:09.369994 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:39:09.369963    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7496565bcf-r9dth_fb9ff216-796a-44c4-95ee-ce96085894f1/manager/0.log"
Oct 30 19:39:35.532665 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:39:35.532627    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:39:44.016439 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:39:44.016405    4544 scope.go:122] "RemoveContainer" containerID="7dbd871c02f071f27552dd43c18fcf5099c46a17db64702900b978f3a3c75817"
Oct 30 19:39:50.374508 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:39:50.374479    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7496565bcf-r9dth_fb9ff216-796a-44c4-95ee-ce96085894f1/manager/0.log"
Oct 30 19:39:50.961239 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:39:50.961204    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7496565bcf-r9dth_fb9ff216-796a-44c4-95ee-ce96085894f1/manager/0.log"
Oct 30 19:40:02.585248 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:02.585213    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-b4fj5_f51455e1-dbba-4186-adbf-60f44c9edbe5/global-pull-secret-syncer/0.log"
Oct 30 19:40:05.532486 ip-10-0-140-122 kubenswrapper[4544]: E1030 19:40:05.532449    4544 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6zrww" containerName="extractor"
Oct 30 19:40:05.587733 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:05.587694    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-r52tw_299ac5c9-51d1-4903-b7f1-2001aa7b2bcf/konnectivity-agent/0.log"
Oct 30 19:40:07.405521 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:07.405489    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-122.us-west-2.compute.internal_7018e001fd99c1f4ae76683b64500513/haproxy/0.log"
Oct 30 19:40:10.435462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:10.435433    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d6dd76dc-q9wt9_59f3ebf1-4b49-4789-bb7c-cca228cfc72a/manager/0.log"
Oct 30 19:40:11.326349 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.326315    4544 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"]
Oct 30 19:40:11.326761 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.326745    4544 cpu_manager.go:414] "containerMap: removing container" podUID="5c426d1d-63bf-4145-99f0-7e08baef8a9e" containerName="maas-api"
Oct 30 19:40:11.326820 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.326762    4544 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c426d1d-63bf-4145-99f0-7e08baef8a9e" containerName="maas-api"
Oct 30 19:40:11.326861 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.326838    4544 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c426d1d-63bf-4145-99f0-7e08baef8a9e" containerName="maas-api"
Oct 30 19:40:11.329969 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.329945    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:11.378894 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.378850    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cx52t\"/\"kube-root-ca.crt\""
Oct 30 19:40:11.378894 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.378857    4544 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cx52t\"/\"openshift-service-ca.crt\""
Oct 30 19:40:11.392425 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.392396    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrlvv\" (UniqueName: \"kubernetes.io/projected/11411c95-276d-4efa-97cc-7f4f2beda13e-kube-api-access-vrlvv\") pod \"perf-node-gather-daemonset-cdxpz\" (UID: \"11411c95-276d-4efa-97cc-7f4f2beda13e\") " pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:11.392561 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.392457    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/11411c95-276d-4efa-97cc-7f4f2beda13e-proc\") pod \"perf-node-gather-daemonset-cdxpz\" (UID: \"11411c95-276d-4efa-97cc-7f4f2beda13e\") " pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:11.392561 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.392534    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/11411c95-276d-4efa-97cc-7f4f2beda13e-podres\") pod \"perf-node-gather-daemonset-cdxpz\" (UID: \"11411c95-276d-4efa-97cc-7f4f2beda13e\") " pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:11.392674 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.392615    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11411c95-276d-4efa-97cc-7f4f2beda13e-sys\") pod \"perf-node-gather-daemonset-cdxpz\" (UID: \"11411c95-276d-4efa-97cc-7f4f2beda13e\") " pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:11.392674 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.392645    4544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11411c95-276d-4efa-97cc-7f4f2beda13e-lib-modules\") pod \"perf-node-gather-daemonset-cdxpz\" (UID: \"11411c95-276d-4efa-97cc-7f4f2beda13e\") " pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:11.425462 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.425431    4544 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cx52t\"/\"default-dockercfg-76gpr\""
Oct 30 19:40:11.427119 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.427091    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"]
Oct 30 19:40:11.493680 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.493650    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrlvv\" (UniqueName: \"kubernetes.io/projected/11411c95-276d-4efa-97cc-7f4f2beda13e-kube-api-access-vrlvv\") pod \"perf-node-gather-daemonset-cdxpz\" (UID: \"11411c95-276d-4efa-97cc-7f4f2beda13e\") " pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:11.494087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.493707    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/11411c95-276d-4efa-97cc-7f4f2beda13e-proc\") pod \"perf-node-gather-daemonset-cdxpz\" (UID: \"11411c95-276d-4efa-97cc-7f4f2beda13e\") " pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:11.494087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.493740    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/11411c95-276d-4efa-97cc-7f4f2beda13e-podres\") pod \"perf-node-gather-daemonset-cdxpz\" (UID: \"11411c95-276d-4efa-97cc-7f4f2beda13e\") " pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:11.494087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.493771    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11411c95-276d-4efa-97cc-7f4f2beda13e-sys\") pod \"perf-node-gather-daemonset-cdxpz\" (UID: \"11411c95-276d-4efa-97cc-7f4f2beda13e\") " pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:11.494087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.493796    4544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11411c95-276d-4efa-97cc-7f4f2beda13e-lib-modules\") pod \"perf-node-gather-daemonset-cdxpz\" (UID: \"11411c95-276d-4efa-97cc-7f4f2beda13e\") " pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:11.494087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.493850    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/11411c95-276d-4efa-97cc-7f4f2beda13e-proc\") pod \"perf-node-gather-daemonset-cdxpz\" (UID: \"11411c95-276d-4efa-97cc-7f4f2beda13e\") " pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:11.494087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.493859    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/11411c95-276d-4efa-97cc-7f4f2beda13e-podres\") pod \"perf-node-gather-daemonset-cdxpz\" (UID: \"11411c95-276d-4efa-97cc-7f4f2beda13e\") " pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:11.494087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.493859    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11411c95-276d-4efa-97cc-7f4f2beda13e-sys\") pod \"perf-node-gather-daemonset-cdxpz\" (UID: \"11411c95-276d-4efa-97cc-7f4f2beda13e\") " pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:11.494087 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.493970    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11411c95-276d-4efa-97cc-7f4f2beda13e-lib-modules\") pod \"perf-node-gather-daemonset-cdxpz\" (UID: \"11411c95-276d-4efa-97cc-7f4f2beda13e\") " pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:11.547923 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.547891    4544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrlvv\" (UniqueName: \"kubernetes.io/projected/11411c95-276d-4efa-97cc-7f4f2beda13e-kube-api-access-vrlvv\") pod \"perf-node-gather-daemonset-cdxpz\" (UID: \"11411c95-276d-4efa-97cc-7f4f2beda13e\") " pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:11.642711 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.642640    4544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:11.683856 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:11.683827    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57bb9bfff-plgsv_15e09f1b-51d2-4943-b382-d40f12efb9a0/manager/0.log"
Oct 30 19:40:12.027192 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:12.027161    4544 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"]
Oct 30 19:40:12.030691 ip-10-0-140-122 kubenswrapper[4544]: W1030 19:40:12.030653    4544 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod11411c95_276d_4efa_97cc_7f4f2beda13e.slice/crio-91daab13fba094e8975c8db0939792cb8965bbd4e3f4fac0d641bbcc4d3d71db WatchSource:0}: Error finding container 91daab13fba094e8975c8db0939792cb8965bbd4e3f4fac0d641bbcc4d3d71db: Status 404 returned error can't find the container with id 91daab13fba094e8975c8db0939792cb8965bbd4e3f4fac0d641bbcc4d3d71db
Oct 30 19:40:12.165157 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:12.165124    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz" event={"ID":"11411c95-276d-4efa-97cc-7f4f2beda13e","Type":"ContainerStarted","Data":"91daab13fba094e8975c8db0939792cb8965bbd4e3f4fac0d641bbcc4d3d71db"}
Oct 30 19:40:12.695625 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:12.695574    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-856dd584c5-khfdk_3684a66d-aa78-4d80-8229-4b32be8f7ac2/manager/0.log"
Oct 30 19:40:13.170678 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:13.170636    4544 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz" event={"ID":"11411c95-276d-4efa-97cc-7f4f2beda13e","Type":"ContainerStarted","Data":"8d7cd05881b1651a5a2c25fce80ecb4bfc1640f7dd591ee8638c4d8c20e54b9e"}
Oct 30 19:40:13.170859 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:13.170706    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:13.279710 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:13.279657    4544 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz" podStartSLOduration=2.279642518 podStartE2EDuration="2.279642518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:40:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:13.278191764 +0000 UTC m=+3810.804066860" watchObservedRunningTime="2026-10-30 19:40:13.279642518 +0000 UTC m=+3810.805517625"
Oct 30 19:40:19.184845 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:19.184817    4544 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cx52t/perf-node-gather-daemonset-cdxpz"
Oct 30 19:40:23.054389 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:23.054316    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sgqww_a25f2140-3a18-44d6-9cfa-7728f9284fee/node-exporter/0.log"
Oct 30 19:40:23.462483 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:23.462410    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sgqww_a25f2140-3a18-44d6-9cfa-7728f9284fee/kube-rbac-proxy/0.log"
Oct 30 19:40:23.869941 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:23.869914    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sgqww_a25f2140-3a18-44d6-9cfa-7728f9284fee/init-textfile/0.log"
Oct 30 19:40:25.535576 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:25.535550    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a13bc23a-de14-4bdd-abae-2b4a146d08c2/prometheus/0.log"
Oct 30 19:40:25.942033 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:25.941964    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a13bc23a-de14-4bdd-abae-2b4a146d08c2/config-reloader/0.log"
Oct 30 19:40:26.349626 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:26.349572    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a13bc23a-de14-4bdd-abae-2b4a146d08c2/thanos-sidecar/0.log"
Oct 30 19:40:26.756356 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:26.756248    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a13bc23a-de14-4bdd-abae-2b4a146d08c2/kube-rbac-proxy-web/0.log"
Oct 30 19:40:27.163526 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:27.163483    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a13bc23a-de14-4bdd-abae-2b4a146d08c2/kube-rbac-proxy/0.log"
Oct 30 19:40:27.613686 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:27.613641    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a13bc23a-de14-4bdd-abae-2b4a146d08c2/kube-rbac-proxy-thanos/0.log"
Oct 30 19:40:28.022949 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:28.022924    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a13bc23a-de14-4bdd-abae-2b4a146d08c2/init-config-reloader/0.log"
Oct 30 19:40:28.435049 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:28.434981    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-p4gdb_4cbe37d7-4720-46d0-9beb-f49960526365/prometheus-operator/0.log"
Oct 30 19:40:28.839224 ip-10-0-140-122 kubenswrapper[4544]: I1030 19:40:28.839198    4544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-p4gdb_4cbe37d7-4720-46d0-9beb-f49960526365/kube-rbac-proxy/0.log"